STATISTICAL MODELING TO INFORMATION RETRIEVAL FOR
SEARCHING FROM BIG TEXT DATA AND HIGHER ORDER
INFERENCE FOR RELIABILITY
XIAOFENG ZHOU
A DISSERTATION SUBMITTED TO THE FACULTY OF GRADUATE
STUDIES
IN PARTIAL FULFILMENT OF THE REQUIREMENTS
FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
GRADUATE PROGRAM IN MATHEMATICS AND STATISTICS
YORK UNIVERSITY
TORONTO ONTARIO
JULY

c Xiaofeng Zhou
Abstract
This thesis examined two research projects probabilistic information retrieval modeling
and thirdorder inference on reliability
In the rst part of this dissertation two research topics in the information retrieval
are carried out and experimented on largescale text data set First we
conduct an indepth study of relationship between information of document length
and document relevance to user need Two statistical methods are proposed which
incorporates document length as a substantial weighting factor to achieve higher retrieval
performance Second we utilize the property of survival function to propose
a costbased reranking method to promote ranking diversity for biomedical information
retrieval and to model the proximity between query terms to improve retrieval
performance Through extensive experiments on standard TREC collections our
proposed models perform signicantly better than the classical probabilistic information
retrieval models
In the second part of this dissertation a small sample asymptotic method is proii
posed for higher order inference in the stressstrength reliability model R  PY
X where X and Y are independently distributed A penalized likelihood method
is proposed to handle the numerical complications of maximizing the constrained
likelihood model Simulation studies are conducted on two distributions Burr type
X distribution and exponentiated exponential distribution Results from simulation
studies show that the proposed method is very accurate even when the sample sizes
are small
iii
Acknowledgements
I would like to express my sincerest appreciation to my thesis supervisors Professor
Augustine Wong and Professor Jimmy Huang whose persistent encouragement
patience guidance and continuous support to lead me into a promising area of research
I am deeply grateful to them their enthusiasm inspirations and great eorts
to guide and help me to approach research projects throughout my PhD study
I am heartily thankful to my supervisory committee member Professor Steven
Wang who is always generously oered his knowledge and help and make my doctoral
study smooth and rewarding
A special thank to all my colleagues from the department of mathematics and
statistics and information retrieval and knowledge management research lab for their
personal support interesting discussions and valuable suggestions
Finally I wish to express my love and gratitude to my husband my parents and
my brothers family for their support care and understanding To them I dedicate
this dissertation
iv
Table of Contents
Abstract ii
Acknowledgements iv
Table of Contents v
List of Tables ix
List of Figures xiii
Introduction and Literature Survey in Information Retrieval
Background of Probabilistic Modeling
Diversication in Biomedical Search
Term Proximity
Summary
Impact of Document Length on Relevance
v
Density Analysis and Length Relevance Weighting
Kernel Density Analysis
Mixture Model
Data Transformation
Distribution of Document Length
Length Relevance Weighting
Evaluation and Experiment Results
The TREC Test Collections
Evaluation Methodology
Comparison with BM
Impact of Parameters
Survival Approach to Diversity and Proximity
Survival Analysis
Survival Approach to Diversity
The Process of Reranking
Experiment Settings and Evaluation Measures
Information Retrieval Baseline Runs
Experimental Results on Genomics collections
Impact of the Survival Models
vi
Experiment Results on Web Track
Survival Approach to Proximity
Reranking Model Based on Term Proximity
Term Proximity Modeling
Experimental Setup
Experimental Results
Conclusion and Future Work in IR
Literature Survey on Asymptotic Methods
Standard Firstorder Likelihoodbased Asymptotic Methods
Saddlepoint Approximation
The Thirdorder Asymptotic Methods
ThirdOrder Likelihood Inference for a General Statistical Model
Canonical Exponential Model
General Exponential Family
General Statistical Model
Computational Issues
Computation Problem
A Penalized Likelihood Method
vii
Application to StressStrength Reliability
Inference for StressStrength Reliability with Burr Type X Distributions
Burr type X Distribution
StressStrength Reliability with Equal Scale Parameters
StressStrength Reliability with Unequal Scale Parameters
Inference for StressStrength Reliability with Exponentiated Exponential
Distribution
StressStrength Reliability with Equal Scale Parameters
StressStrength Reliability with Unequal Scale Parameters
Conclusion and Future Work on Thirdorder Asymptotic Methods
Bibliography
viii
List of Tables
Information about the test collections
Evaluation results over the BM baseline with b   A star indicates a
statistically signicant improvement over the baseline
Evaluation results over the BM baseline with optimized setting of b A
star indicates a statistically signicant improvement over this baseline
Evaluation results over the BM baseline on disak
The best and mean results in the Genomics tracks
Reranking Performance based on Aspect Detection Using Wikipedia
Reranking Performance based on Aspect Detection Using UMLS
Reranking Performance on the Diversity Task of TREC  Web
Track
The MAPstatMAP values obtained by the unigram BM baseline
and the bigram model
ix
The MAP values obtained by the unigram BM baseline and the
bigram model on WTG
The MAP values obtained by the unigram BM baseline and the
bigram model on Blog
The MAP values obtained by the unigram BM baseline and the
bigram model on GOV
The statMAP values obtained by the unigram BM baseline and the
bigram model on ClueWeb B
The MAPstatMAP values obtained for evaluating the trigram model
A star indicates a signicant improvement over the unigram BM
baseline
Three simulated data sets
central condence intervals for
x      x  y
x
y  c
x n   and m
x      x  y
x
y  c
x n   and m
x      x  y
x
y  c
x n   and m
The strength data for single carbon bre at mm mm
Parameter Estimates for Example using iterative and penalized method
Parameter Estimates for Example when N    and
x
Parameter Estimates for Example when K     and
with N
Interval Estimates of   for Example assuming equal scale parameter
nm    and  satises R
nm    and  satises R
nm    and  satises R
Interval Estimates of   for Example assuming unequal scale parameters
Interval Estimates of   for Example using resampling
Simulation results for R when  is not equal
and  is obtained by Equation
nm
and  is obtained by Equation
nm
and  is obtained by Equation
nm
Interval Estimates of   Assuming EED with Same Scale Parameter
and  satises R     nm
and  satises R     nm
and  satises R     nm
xi
Interval Estimates of   for Example
and  is obtained by Equation
nm
and  is obtained by Equation
nm
and  is obtained by Equation
nm
xii
List of Figures
A basic IR system
Illustrate two hypotheses using three extreme cases
Kernel density estimate constructed from the document length on four
test collections dl stands for document length
Kernel density estimates constructed from the transformed document
length of four test collections rel stands relevant document length
and non stands nonrelevant document length
Distribution ttings for the relevant document length using transformation
two z  logd
Performance of BML over BM
The MAPstatMAP values obtained against the parameter b
The MAPstatMAP values obtained against the parameter
xiii
Eects of Survival Model and Aspect Filtering on s topicsThe
xaxis presents the evaluation measures where NLM MuM and
Oka in the left gure stand for three baselines corresponding to
NLMinter MuMshFd and Okapi
Eects of Survival Model and Aspect Filtering on s topicsThe
xaxis presents the evaluation measures where a b and c
in the right gure stand for three baselines corresponding to Okapia
Okapib and Okapic
p for Data Set
p for Data Set
p for Data Set
xiv
Introduction and Literature Survey in
Information Retrieval
Information retrieval IR is the science of searching for documents for information
within documents and for metadata about documents as well as that of searching
relational databases and the World Wide Web Generally an IR system receives a
query from user and returns the supposedly relevant documents where a query is
a statement of an information need A crucial issue underlying an IR system is to
rank the returned documents by decreasing order of relevance In most traditional
retrieval systems queries are translated into query representations Similarly documents
are converted into document representations Figure  describe a basic IR
system in which the IR model proposes to match the query representation against
the document representation and computes a numeric score on how relevant each
document representation satises the users query and then ranks the documents
according to their scores Finally the document representations are recovered to be

the original documents for users reading
Users
Query
Query Representation
IR Models
Retrieved
Relevant
Documents List
Retrieved
Relevant
Documents
Documents
Collections
Documents
Representation
Figure  A basic IR system
In general ranking is based on a weighting model In Section  we review the
basic probabilistic model which is one of the most popular weighting models in modern
IR systems and its development Retrieved documents are ranked in the order
of relevance to the query A good IR system should return ranked lists that respect
both the queryrelevance and the breadth of available information Section  introduce
the most recent research on promoting search result diversication in ranked
document lists Utilizing term proximity to improve retrieval performance is given
in Section  Finally some concluding remarks are given in

Background of Probabilistic Modeling
The probabilistic approach in IR is extensively studied in the literature this family
of IR models is developed by Robertson  from the Probability Ranking
PrinciplePRP Cooper  gives the formal statement of PRP as follows
Probability Ranking Principle  If a reference retrieval systems response to
each request is a ranking of the documents in the collection in order of decreasing
probability of relevance to the user who submitted the request where
the probabilities are estimated as accurately as possible on the basis of whatever
data have been made available to the system for this purpose the overall
eectiveness of the system to its user will be the best that is obtainable on the
basis of those data
The basic weighting function proposed by Robertson et al  is one of the most
popular weighting models in modern IR systems and it can be expressed as follows
wX  log
PXjR PjR
PXjR PjR

where X is an information vector about the document  is a reference vector representing
a zeroweighted document and R and R are relevance and nonrelevance
respectively This model considers the independent assumption of the query term
within documents made by Robertson and Jones  At this stage X contains

the information about term frequency within documents only So it is so natural
to incorporate the document length into this consideration Robertson and Walker
introduced the document length d into the basic probabilistic weighting model
in Equation  and rened it based on the two hypotheses Verbosity and Scope hypotheses
Verbosity hypothesis given by Robertson andWalker  states that the
document length is independent from its relevance In other words long documents
simply use more words than short documents to cover similar scope An opposite assumption
about document length is the socalled Scope hypothesis which states that
some documents may contain more material than others if longer more details can
be found at Robertson and Walker  That is long documents are more likely
to be retrieved In practice a document may be considered as a tradeo between
the Verbosity hypothesis and the Scope hypothesis How to balance between these
two hypotheses by modeling document length within the basic probabilistic weighting
paradigm remains a challenging research issue The impact of document length
on relevance is particularly important for adhoc retrieval where relevance is dened
in a binary or graded manner Compared to a short document a long document is
likely to be relevant if it contains paragraphs that meet the information need of the
query even if a large part of the document is in fact nonrelevant Figure  elaborates
the idea of two hypotheses with the three extreme cases For example if the

Earthquake
in Japan
Users Query
Verbosity
Hypothesis
Earthquake
in Japan
A massive
magnitude
quake hit northeast
Japan on Friday
Extreme Case
Earthquake in Japan
A massive magnitude quake
hit northeast Japan on Friday
A massive magnitude quake
hit northeast Japan on Friday
Japan was hit by a strongest
earthquake on Friday
A massive magnitude quake
hit northeast Japan on Friday
Extreme Case
Scope
Hypothesis
Earthquake in Japan
A massive magnitude quake
hit northeast Japan on Friday
Janpans meteorological agency is
sued a tsunami warning
Japan passes budget for emergency earthquake relief
The economic impact of the earthquake
According to Reuters Fukushima nuclear plant workers
evacuated to higher ground following the earthquake
Japans earthquake and tsunami devastated the country
and wreaked havoc at the Fukushima nuclear plant
Quake hits central Japan no nu
clear reactor problem reported
The March  earthquake caused extensive
damage at the Fukushima nuclear plant
Four months ago the same area
was hit by a massive earthquake
Extreme Case
Earthquake in Japan
A massive magnitude quake hit northeast Japan on Friday
A massive magnitude quake hit northeast
Japan on Friday causing dozens of deaths
Janpans meteorological agency issued a tsunami
warning after the earthquake in May
Japan passes budget for emergency earthquake relief
The economic impact of the earth
quake and Japans struggle to recover
According to Reuters Fukushima nuclear plant workers
evacuated to higher groundfollowing the earthquake
Mixture



Figure  Illustrate two hypotheses using three extreme cases

users query is Earthquake in Japan we can nd a short document with only one
sentence A massive magnitude quake hit northeast Japan on FridayExtreme
Case  and a long document with repeated sentences A massive magnitude
quake hit northeast Japan on Friday or similar scope but other words like Japan
was hit by a strongest earthquake on FridayExtreme Case  Verbosity hypothesis
says the document length is independent from its relevant so these two documents
should be weighted the same but clearly document in extreme case  is longer than
the one in extreme case  Figure  gives a typical example of Scope hypothesis
extreme case  A document with a lot of more and dierent information related
to users query Earthquake in Japan but has same document length as extreme
case  It may repeated same piece of information several times but it do contain
more information than the previous two cases Scope hypothesis says that we should
weight it more than the previous two cases but Verbosity hypothesis says opposite
Robertson and Walker  recite that real document collections contains
a mixture of eects from two hypothesis and the individual document may be at
either extreme case or the combination Normally in a collection of a documents
the document length is varying may from couple of words to thousands or millions
words One may think the more likely the document contain the query term if the
document length is more longer It has been recognized as an important factor for

adjusting the IR system to avoid the length bias but the impact of document length
on relevance remains challenging
To address the eect of document length on relevance the basic probabilistic
weighting function with document length been taken into account by Robertson and
Walker  becomes
wX d  log
PX djR
PX djR
PjR
PjR

where wX d is the relevance weight of a given document d is the document
evidence for relevance which is given by document length  denotes the average
document length of the reference vector  and X represents all other information
about the document R and R stand for the nonrelevance and relevance respectively
This function measures the dierence between the probabilities of document length
and all other information we have for the document when it is relevant and when it
is not relevant respectively in log scale The above equation also implies a relevant
document should receive a higher weight than a nonrelevant document in order to
achieve a satisfying retrieval performance
Equation  can be further decomposed into the three components as follows
Robertson and Walker
wX d  wX d  w d  wd
where

wX d  log
PX djR
PX djR
P djR
P djR

w d  log
PjdR
PjdR
PjR
PjR

and
wd   log
PdjRPjR
PdjRPjR

Under the Verbosity hypothesis document length has been considered as independent
evidence of relevance This hypothesis nullies the component wd  in
Equation  which as a consequence is set to zero in Robertson and Walker
Thus the weighting function becomes
wX d  wX d  w d
The classical BM weighting model derived by HancockBeaulieu et al  from
Equation  more specically wBM  wX d where wBM is the relevance
score of BM given by the following weighting function
wBM
k  tf
K  tf
w
k  qtf
k  qtf
L
where
K  k

b  b
dl
avdl


w  log
r  R  r
n  r  N  n  R  r

L  k  nq
avdl  dl
avdl  dl


N is the number of indexed documents in the collection n is the number of documents
containing the query term R is the number of known relevant documents to a specic
topic r is the number of relevant documents containing the term tf is withindocument
term frequency qtf is withinquery term frequency dl is the document
length ie the document evidence d in Equation  avdl is the average document
length nq is the number of query terms ki
s and b are tuning constants whose
setting depends on the dataset used and is usually empirically determined and
indicates that its following component is added only once per document Particularly
b functions as a justication factor that adjusts the relative importance between the
two hypotheses given by Robertson et al
The classical probabilistic models for IR rank documents according to their relevance
scores assigned by matching the query terms with adjustment for the relationship
between document length and term frequency This approach is developed based
on the Verbosity hypothesis which assumes the documents relevance is independent
of its length However in practice the impact of document length on relevance may
be a mixture of both the Scope hypothesis and the Verbosity hypothesis brought by
Robertson and Walker  Many previous studies have been conducted to investigate
the impact of document length on relevance Singhal et al a suggested
that long documents tend to have more unique terms and consequently long docu

ments have a better chance to be retrieved than short documents As the document
length increases the number of times the query terms occur in the documents also
increases which in turn increases the matching score For instance Singhal et al
ab illustrated that the probability of a documents relevance increases proportionally
with document length in the early TREC test collections and showed that
better retrieval performance can be achieved with normalization techniques They
also reported that documents retrieved by a model produce a retrieval pattern by
the distribution of the document length Moreover a number of empirical studies
have provided statistical evidence supporting that the probability of a documents
relevance to an information need is considered to be correlated with the length of the
document Kraaij et al  showed that the probability of relevance is positively
correlated with document length on a number of TREC adhoc and Web collections
Furthermore Voorhees et al  found that proper term weighting strategies
based on document length can also improve retrieval performance For example
normalization techniques have been applied for each term in the query through the
length adjustment to avoid the bias introduced by document length Losada and
Azzopardi  applied JelinekMercer and Dirichlet prior to twostage smoothing
strategies on document length in language modeling to show the signicant impact
of document length on the information retrieval performance

Diversication in Biomedical Search
Traditional retrieval models assume that the relevance of a document is independent
of the relevance of other documents as stated in Section  However in reality this
assumption may not hold This assumption may result in high redundancy and low
diversication in a ranked document list since documents that are similar in content
tend to appear over and over again That is novel information is needed
The usefulness of retrieving a document usually depends on previous ranked documents
since a user may want to see the top ranked documents concerning dierent
aspects of hisher information need instead of reading relevant documents that only
deliver redundant information A better information retrieval system thus should
return ranked lists that represent both the queryrelevance and the breadth of available
information For example in biomedical domain the desired information of
a question query asked by biologists usually is a list of a certain type of entities
covering dierent aspects that are related to the question such as genes proteins
diseases mutations etc Hence it is important for a biomedical IR system to be able
to provide comprehensive and diverse answers to fulll biologists information needs
The dissimilarity between documents has to be considered In the TREC  and
Genomics tracks the aspect retrieval was investigated Its purpose was to
study how a biomedical IR system can support a user gather information about the

dierent aspects of a topic In the Genomics tracks biomedical IR systems were
required to return relevant information at the passage level while relevance judges
not only rated the passages but also grouped them by aspect Aspects of a retrieved
passage could be a list of named entities or MeSH terms representing answers that
cover dierent portions of a full answer to the query Aspect Mean Average Precision
Aspect MAP was dened in the Genomics tracks to capture similarities and
dierences among retrieved passages It indicates how comprehensive the questions
are answered Hersh et al a found that relevant passages that do not contribute
any new aspects to the aspects retrieved by higher ranked passages will not
be used to accumulate Aspect MAP Therefore Aspect MAP is a measurement for
redundancy and diversity of the IR ranked list
Our work is inspired by several recent papers that concerned with promoting diversity
and novelty in the IR ranked list Carbonell and Goldstein  introduced
the maximal marginal relevance MMR method which attempted to maximize relevance
while minimizing similarity to higher ranked documents In order to measure
the redundancy between documents Zhang et al  presented four redundancy
measures which were set dierence geometric distance distributional similarity
and a mixture model They modeled relevance and redundancy separately
Since they focused on redundant document ltering experiments in their study were

conducted on a set of relevant documents However in reality nonrelevant documents
are always returned by IR systems along with relevant documents Redundancy
and relevance should both be considered Zhai et al  validated a
subtopic retrieval method based on a risk minimization framework Their subtopic
retrieval method combined the mixture model novelty measure with the query likelihood
relevance ranking More recently a new diversity task of Web retrieval was
dened in the TREC  Web track by Clarke et al a Two evaluation
measures nDCG by Clarke et al a and an intentaware version of precision
IAP by Agrawal et al a both of which reward novelty and diversity
were validated in the diversity task of the  Web track Top diversity task results
showed that reranking methods based on anchor text sites of search results
link ltering clustering and subqueries suggestion were eective in Web retrieval
result diversication More details can be found atSong et al  Craswell et al
Kaptein et al  and McCreadie et al  Santos et al b proposed
a novel framework namely xQuAD for search result diversication that builds such a
diversied ranking by explicitly accounting for the relationship between documents
retrieved for the original query and the possible aspects underlying this query in
the form of subqueries Their study showed that the subquery generation step
plays a fundamental role in the method When they use the groundtruth subtopics

provided by the collection of TREC Web track as input to their proposed diversi
cation models performance improvements can be obtained Howeverwhen they
use a clusteringbased query expansion technique in an attempt to uncover terms
representative of dierent aspects underlying a query from a clustering of the top
retrieved results for the query no consistent performance improvements can be observed
Santos et al a also used three major web search engines WSE to
generate the subqueries and investigated the impact of subquery importance In
general using the related or suggested queries from WSEs improvements over the
initial ranking are obtained in most settings
In biomedical information retrieval the Genomics aspect retrieval was rstly
proposed in the TREC  Genomics track and further investigated in the
Genomics track Many research groups joined these annual campaigns to evaluate
their systems and methodologies However to the best of our knowledge there is not
too much previous work conducted on the Genomics aspect retrieval for promoting
diversity in the ranked list
Term Proximity
In literature term proximity is interpreted as query term cooccurrences or phrases
Proximity searches for documents where two or more separately matching term oc

currences are within a specied distance where distance is the number of intermediate
words or characters Proximity among query terms has been found to be useful
for improving retrieval performance The intuition behind the development of the
proximitybased models is that documents in which query terms cooccur in a close
proximity or within the same phrases tend to be highly relevant
Term proximity is particularly useful in Web environment where documents are
highly diverse and heterogeneous Indeed some previous studies have demonstrated
the eectiveness of using term proximity for improving the retrieval performance
Such research has resulted in the development of the socalled ngram models in
contrast to the classical unigram models which assume term independence An
ngram is referred to as a subsequence of n terms from a given sequence or a given
window of terms In addition the ngram terms are not necessarily adjacent to each
other For example if an ngram modeling proximity appears in Modeling term
proximity for probabilistic models the two ngram terms have a distance of  since
term appears between modeling and proximity in the text
In the past decades there have been attempts to develop more sophisticated
IR models by employing the term proximity information Fagan  empirically
identied phrases using features such as the frequency of the phrase in the collection
and the proximity of the phrase terms and concludes that the tested phrase

methods do not perform consistently well across a variety of collections Croft et al
automatically extracted phrases from natural language queries to form structured
queries for a probabilistic ranking model van Rijsbergen  proposed a
theoretical model for incorporating term cooccurrence information to a binary independence
model Losee and Jr  applied the Bahadur Lazarsfeld expansion to
identify term dependence between more than two terms In addition Yu et al
generalized both the tree dependence model and the Bahadur Lazarsfeld expansion
Although the two methods are of high complexity in theory their evaluation showed
little improvement over a unigram model argued by Metzler and Croft a
Recently the application of term proximity has achieved a certain degree of success
in the context of the language modeling approach Gao et al  incorporated
the term dependence based on a link structure for each query which could be timeconsuming
in practice Metzler and Croft a proposed a Markov Random Field
model to estimate term dependencies in the context of language modeling approach
A similar idea was presented by Mishne and de Rijke  Lv and Zhai
applied a list of kernel functions to estimate a language model for every position
in a document Zhao and Yun  constructed the query term proximity as the
Dirichlet hyperparameter that weights the parameters of the unigram document language
model The application of term proximity to the probabilistic models has little

success so far Plachouras and Ounis a conducted an initial study on incorporating
term proximity in the Divergence From Randomness models They assume a
Poisson distribution of ngram cooccurrence in the elite set ie the set of retrieved
documents However the approach proposed by Plachouras and Ounis a only
leads to a moderate improvement over the unigram baseline possibly due to the lack
of the collection model for the ngram frequency distribution Bttcher and Clarke
Rasolofo and Savoy  also attempted to incorporate the term proximity
into the classical BM model which dene the bigram cooccurrences as an inverse
function of the square distance between two query terms However the retrieval
performance reported by Bttcher and Clarke  Rasolofo and Savoy  is
not as good as expected possibly due to the naive bigram occurrence function
Summary
I brie
y reviewed the basic weighting model in this Chapter With its recent development
in IR eld my research project focus on three parts In Chapter  I study the
impact of document length on its relevance in the context of the Scope hypothesis
In Chapter  I will show that retrieval performance can be improved by applying
survival function on aspect and term proximity Finally in Chapter  I will discuss
possible future direction of this research

Impact of Document Length on Relevance
The main focus of this research project is to study the impact of the document length
on its relevance The Scope hypothesis presented by Robertson and Walker
suggests the existence of a relationship between document length and relevance It
implies that the component wd  in Equation  may not be zero In this
chapter we consider document length itself as a direct predictor of relevance The
study of document length is based on the intuition that long documents tend to have
high retrieval probabilities since long documents usually have a large number of
unique terms which are likely to be picked up by the query term matching Singhal
et al a In Section  we study the pattern of the document length and
its relevance by exploring a list of probability density distribution investigate the
behavior of wd  in Equation  a new weighting function incorporating this
relationship proposed in Section  which is the result of a mixture of the two
hypotheses Our results presented in Section  show that the retrieval performance
of BM can be markedly improved over dierent settings of the parameter b in

Equation  by exploiting the document length evidence More details can be
founded in Zhou et al
Density Analysis and Length Relevance Weighting
Under the Scope hypothesis wd  in Equation  is no longer zero since a dependence
of relevance on document length is assumed To add the length information
into the weighting function wx d we decompose the wd  further into
wd   log
PdjR
PdjR
log
PjR
PjR

The second component of Equation  is constant over a given document collection
This is because the average document length  for the reference vector  in a
document collection is known and xed Therefore for each document in a collection
the second component of Equation  above is the same across the whole document
collection and does not aect the document ranking For simplicity we refer
wd  to as the rst component in the Equation  Thus the relevance weight
wd  is given by the log odds ratio of the relevance and nonrelevance probabilities
PdjR and PdjR In other words wd  measures the dierence between
the probabilities of given document length condition on relevance and nonrelevance
in log scale We name Equation  as length relevance weighting Our ultimate
goal is to calculate the wd  in Equation  this needs a way to estimate the

probabilities PdjR and PdjR By adding the measurement of document length
itself into the basic weighting function the retrieval system is expected to achieve
high accuracy since the length information brings more evidence of relevance The
estimation of probability distribution function  of document length will be discussed
in the next subsections
For the rest of this chapter we use d to denote the document length As a
general rule we usually make an assumption about observed ds ie d d  dN
are independent and identically distributed N is the number of documents in the
collection For example N is  in test collection disk The proposed
method will be examined on the four standard TREC test collections disk
WTG GOV and ClueWeb B The detailed introduction on four test collections
will be given in the Section
Kernel Density Analysis
Kernel density estimation or Parzen window method is a nonparametric way of
estimating the probability density function of a random variable It can be used to
Probability distribution function is a general concept in probability theory it could refer to
probability mass function on discrete random variable probability density function on continu
ous random variable or cumulative distribution function based on context So we use probability
distribution function and probability density function interchangeable in the rest of this paper

extrapolate data to the entire population In particular Silverman  denes
b fhd

nh
Xn
i
K

d  di
h


where di i       n is the independent and identicallydistributed sample from
some unknown distribution n is the number of samples we draw from the population
K is the kernel function and h is the bandwidth also known as the smoothing
parameter We can obtain the smoothing curve by adjusting the parameter h
Usually K is a standard Gaussian function with a mean of zero and a variance of
K

d  di
h



p

exp


d  di
h


Kernel density estimation gives us a global picture of the given dataset
Figure  shows the distributional pattern of relevant and nonrelevant document
length for the test collections disk GOV WTG and ClueWeb B respectively
using kernel density estimation with the standard normal kernel function In Figure
the length curves of rst three subgures have been substract when the document
length is large than  in xaxis in order to visualize the dierence between
the relevant and nonrelevant document length because the long documents have
low frequencies to be emerged in test collections The length of nonrelevant and
relevant documents from four test collections are both positively skewed and have
a long tail with dierent tail shapes Nonrelevant document has higher frequency












x
Density estimate
nonrelevant dl
relevant dl
a disk









x
Density estimate
nonrelevant dl
relevant dl
b WTG











x
Density estimae
nonrelevant dl
relevant dl
c GOV








x
Data
Density estimate
nonrelevant dl
relevant dl
d ClueWeb B
Figure  Kernel density estimate constructed from the document length on four
test collections dl stands for document length

than relevant documents when the document length is relatively short The curve
on disk appears to have two distinct peaks called bimodality In contrast there
is only one mode that arises on WTG GOV and ClueWeb B
Mixture Model
The mixture model is a probabilistic model for representing the presence of more
than one subpopulations within an overall population Gaussian Mixture Models
GMMs are among the most statistically mature methods for density estimation A
Gaussian mixture model is a weighted sum of M component Gaussian densities as
given by
fdj     M    M      M
XM
m
m gdjmm
where m is the weight of each component and
PM
m m   mm are the parameters
of Gaussian distribution gm which takes form of
gdjmm

Vjmj exp




d  m
m d  m


Each Gaussian has its own mean m and covariance matrix m V is the dimension
of d For mixture model data within same group assumed be homogeneous tends to
tight data between groups are heterogeneous All component Gaussian are acting
together to model the overall feature density so mixture model covers the data well

The GMMs not only provide a smooth overall distribution t its components also
clearly detail the multimodal nature of the density Figure  exhibits that there
are two modes on disk The parameters of GMMs can be estimated by the
maximum likelihood ML criterion using the iterative ExpectationMaximization
EM algorithm
Data Transformation
The data transformation technique is widely used in data processing or preprocessing
for stabilizing the variance and make the data more normal distributionlike In our
case all document lengths are positive whose distribution is skewed to the right
as described in Figure  and document length cannot be described by standard
statistical methods because of the skewness Therefore data transformation is required
to extract a better characteristic of the data We start with standardization
the transformation I By standardizing the data it forces the data to locate on the
common scales to be compared Secondly the BoxCox transformation is from the
family of functions that are applied to create a rankpreserving transformation of
data which improves the correlation between variables and for other data stabilization
procedures Box and Cox  BoxCox transformation is commonly used to
alleviate heteroscedasticity when the distribution of the variable of interest is not

known ie transformations II and III We also transform the document length to be
within the scale of    using transformation IV The four types of transformation
is described as follows
Transformation I Standardization
z
d   d
sd

Transformation II Log transformation It is the special case of following transformation
III when
z  logd
Transformation III BoxCox transformation
z
d


Boxcox transformation is a parametric power transformation technique in order
to reduce anomalies such as nonadditivity nonnormality and heteroscedasticity
is the transformation power
Transformation IV    normalization
z
d  dmin
dmax  dmin


where z is the document length after the transformation  d is the average document
length sd is the standard deviation of document length dmin denotes the minimum
document length and dmax is the maximum document length
Figure  plots the distributional pattern of transformed relevant and nonrelevant
document length on four test collections In Figure  we examine length
distribution patterns of relevant and nonrelevant documents on the four test collections
used A major observation is that the the curves of the transformed document
length distribution have similar shapes before and after the transformation That is
the curves on disk remain bimodal while the curves on the other three test collections
are still leftskewed but not as much as those of the original document length
distribution thanks to the data transformation Moreover the curves on GOV and
ClueWeb B become more symmetric after the transformation On disk GOV
and WTg the center of the nonrelevant document length distribution shifts far
away to the right of the relevant document length distribution From Figure d
we can clearly see that relevant and nonrelevant document length on Clueweb B
can be distinguished from their distributional frequencies Note that similar observations
can also be drawn from other three collections used but the dierence
between relevant and nonrelevance document length distribution is not as obvious
as on ClueWeb B This is an encouraging nding as it gives us clue of dierentiating








Density
diskrel
disknon
GOVrel
GOVnon
WTGrel
WTGnon
ClueWebBrel
ClueWebBnon
a Transformation I








Density
diskrel
disknon
GOVrel
GOVnon
WTGrel
WTGnon
ClueWebBrel
ClueWebBnon
b Transformation II








Density
diskrel
disknon
GOVrel
GOVnon
WTGrel
WTGnon
ClueWebBrel
ClueWebBnon
c Transformation III










Density
diskrel
disknon
GOVrel
GOVnon
WTGrel
WTGnon
ClueWebBrel
ClueWebBnon
d Transformation IV
Figure  Kernel density estimates constructed from the transformed document
length of four test collections rel stands relevant document length and non
stands nonrelevant document length

between relevance and nonrelevant documents based on their length distribution
In the next subsection we propose to t the document length distribution with a
list of statistical distributions in order to nd the distributions that can match the
characteristics of relevance and nonrelevant documents
Distribution of Document Length
The criterion of selecting distributions is that the distribution must be positive
skewed with shape and rate parameters It is impossible to use one single distribution
to capture all kinds features of all dierent document collections The commonly
used distributions we applied to t the transformed document length are as follows
Gamma distribution with

fy
y
ey



for y   where
and  are shape and scale parameters respectively Varying
setting of
can lead to symmetrical or skewed gures
Normal distribution with   which is symmetric with respect to its mean
value  and the variance  measures the width of the distribution A Normal
distribution is bell shaped and the shape is independent of its distribution
parameters The reason of choosing normal distribution is that transformation

I try to standardize the document length The Normal distribution density
function is given as follows
fy

p

exp


y


where  is the standard deviation
Lognormal distribution with
fy

y
p

exp


log y


for y   where if Z is distributed lognormally with parameters  and
logY  is distributed normally with a mean of  and a standard deviation of
Lognormal and gamma distribution can produce similar graphs but the
curvature of lognormal distribution is more steep than gamma distribution
Inverse Gaussian distribution IGD with
fy
s

y exp



y
y

for y   where    is the mean and    is the shape parameter changing
changes the level of the skewness for the IGD
Weibull distribution with a b
fy
b
a
y
a
b
exp
n

y
a
b
o

where a   is the scale parameter and b   is the shape parameter Weibull
distribution can produce the graph similar to Gamma distribution but with
less steep curve
Generalized Extreme Value distribution GEV
fy


exp


y





y



where    is the shape parameter  is the location parameter and
is the scale parameter Compared to the statistical distributions mentioned
above GEV is a complicated distribution developed within the extreme value
theory introduced by Gumbel
Figure  illustrates the six distribution ttings for the relevant document length
of four test collections using Transformation I Similar plots can be obtained for the
relevant and nonrelevant document length of all four test collections using Transformation
I Transformation III and Transformation IV respectively All six distributions
t the GOV WTG and ClueWebB well not disk since the bimodality
Inverse Gaussian and GEV distribution t the data best on all test collections
Weibull distribution can preserve the skewness better than the Lognormal Gamma
distribution but normal distribution performs very badly in this case since skewness
of the data After the density functions are t to the actual length distribution it










Probability Density
document length
Gamma fit
Normal fit
Lognormal fit
Inverse Gaussian fit
Weibull fit
GEV fit
a disk







Probability Density
Document length
Gamma fit
Normal fit
Lognormal fit
Inverse Gaussian fit
Weibull fit
GEV fit
b GOV








Probability Density
Document length
Gamma fit
Normal fit
Lognormal fit
Inverse Gaussian fit
Weibull fit
GEV fit
c WTG










Probability Density
Document length
Gamma fit
Normal fit
Lognormal fit
Inverse Gaussian fit
Weibull fit
Gev fit
d ClueWebB
Figure  Distribution ttings for the relevant document length using transformation
two z  logd

is necessary to use goodness of t test to determine how well the distributions t to
the actual data
Bootstrapping is a resampling method to learn about the sample characteristics
to infer the population It has been proved eective in reducing the bias of samples
by Gentle  Ader et al  recommend to use bootstrapping when the
sample size is insucient for straightforward statistical inference The bootstrapping
procedure is described as follows
Construct an empirical probability distribution
from the sample by placing
a probability of n at each point z z     zn of the sample This is the
empirical distribution function of the sample which is the nonparametric maximum
likelihood estimate of the population distribution  Now each samples
element has the same probability of being drawn
From the empirical distribution function
draw a random sample of size n
with replacement This step is called resampling
Calculate the statistic of interest  for this resample yielding
Repeat steps  and  for B times where B is a large number in order to create
B resamples The setting of B depends on the tests to be run on the data
Compute


B
PB
j

j

Length Relevance Weighting
Based on the discussion above we rst derive the length relevance weighting function
using Equation  for disk when the distribution function obtained from
mixture model We named it Mixture
wd   log
R
R
exp


d  R

R


R
R
exp


d  R

R

NR
NR
exp


d  NR

N R


NR
NR
exp


d  NR

N R

The subscript RRNRNR represent the relevant and nonrelevant for Gaussian
component  and  respectively    are the parameters of GMMs
Second when the distribution function for transformed document length Z is
obtained we apply variable change technique in Shao  to obtain the distribution
function for the original document length D ie the document length before
transformation The theorem of change variable we used is as follows
Theorem  Let Z be a random variable with probability density function pdf
fZz and support SZ Let D  gZ where gz is one to one dierentiable function
on the support of Z SZ Denote the inverse of g by z  gd and let z
d  gd
d
Then the pdf of D is given by
fDd  fZzd

z
d

for d  SD
with the support of D which is the set SD  fd  gz  z  SZg

Where zd is equivalent to Equations    and  when transforming
the document length and j z
d j is the determinant of Jacobian of the transformation
in Hogg et al
Based on the discussion above we initiate the pattern of the document length by
kernel density estimation Based on the ndings in step one second we apply data
transformation and the change variable techniques to nd the distribution functions
of relevant and nonrelevant document length and use maximum likelihood estimation
MLE to obtain the parameter estimators Two statistical methods EM and
bootstrapping are exploited to prevent potential bias during parameter estimation
such as incomplete test collection randomness of sampling Hypothesis test employees
to eliminate the distributions at  signicance Finally Equation  is used
to construct the following seven models
In this model with standardization transformation I in Equation  transformed
relevant and nonrelevant document length follow the Normal distribution
Using Equation  and change of variable technique in Equation
the length relevance weighting function named Normal is as follows
wd




d   d
sd








d   d
sd



where subscript  indicates that it is the estimates of distribution for relevant
document length  is the estimates of nonrelevant document length distribu

tion
In this model with log transformation II in Equation  transformed relevant
and nonrelevant document length follow the Gamma distribution Similarly
the length relevance weighting function named LogGamma is as follows
wd

log d
log d


log d

In this model with log transformation transformed relevant and nonrelevant
document length follow the IGD distribution We call the length relevance
weighting function LogIGD
wd  log
s

log d
log
s

log d

log d

log d

log d

log d
In this model with BoxCox transformation in Equation  transformed
relevant and nonrelevant document length follow the Inverse Gaussian distribution
distribution Then the length relevance weighting function is called
BoxCoxIGD
wd  log
s

d


log d


log d

log
s

d


log d


log d


In this model using BoxCox transformation transformed relevant and nonrelevant
document length follow the GEV distribution Then the length relevance
weighting function is called BoxCoxGEV
wd





log
V




V









log
V




V



where V  d
In this model using normalization transformation in Equation  transformed
relevant and nonrelevant document length follow the Lognormal distribution
Then the length relevance weighting function is called Lognormal
wd



log L



log L
where L  d  dmindmax  dmin
In this model using normalization transformation transformed relevant and
nonrelevant document length follow the Weibull distribution Then the length
relevance weighting function is called Weibull
wd   b  b log L
L
a
b
L
a
b
where L  d  dmindmax  dmin

Based on the discussion above we propose a new lengthbased weighting function
BML as follows
wx d     wBM    wd
where wBM is the relevance score of BM  indicate that the term wd  is
added only once for each document  is not only the interpolation factor which is
empirically determined and highly depends on the dataset used but also an adjust
factor of the mixture of two hypotheses Verbosity and Scope hypothesis A document
could be either extreme or of mixture of these two hypotheses as discussed in
Robertson and Walker  More over the reason of adding  here is that we
ignore the constant term in the calculation of log PjR
PjR  we need to adjust the scale
for the weights between wBM and wx d and the weights between two hypotheses
because BML consider the situation when both Verbosity and Scope hypothesis
are presented in the same document For a given query each of the wBM or wx d
scores is normalized by the maximum wBM or wx d score The parameter  is
obtained by Simulated Annealing in Kirkpatrick et al  over a set of training
topics

Evaluation and Experiment Results
We rst give detail information about the four test collections in subsection
introduce our methodology for evaluating the BML model in Subsection
and the evaluation results comparing with BM baseline is presented in Subsections
The impact of parameters b and  is investigated in Subsection
The TREC Test Collections
We examine the impact of document length on relevance using four standard TREC
test collections These four test collections are the most recent TREC datasets and
provide a good coverage on the a variety of commonly used datasets in IR evaluation
and are used for dierent test purposes and vary in size in term of the document
length Basic information about the test collections and topics are given in Table

Table  Information about the test collections
Coll TREC Task Topics  Docs
disk  Adhoc
WTG   Web
GOV  Terabyte Adhoc
ClueWeb B  Relevance Feedback rfrf
The disk collection contains newswire articles from various sources such as

Association Press AP Wall Street Journal WSJ Financial Times FT etc
which are usually considered as highquality text data with little noise It usually
used for ad hoc test The WTG collection is a medium size crawl of Web documents
which was used in the TREC  and  Web tracks It contains  Gigabytes
of uncompressed data The GOV collection which has  Gigabytes of uncompressed
data is a crawl from the gov domain This collection has been employed
in the TREC     and   Terabyte tracks The ClueWeb
collection is a very large crawl of the Web and is currently the largest TREC test
collection We use the category B of ClueWeb which contains about  million
English Web pages and its associated topics used in the TREC  Relevance
Feedback track We index all documents in the above four collections For all four
test collections used each term is stemmed using Porters English stemmer and
standard English stopwords are removed
Evaluation Methodology
We evaluate our proposed BML model over the  test collections used namely
disk WTG GOV and ClueWeb B Each topic contains three topic elds
namely title description and narrative We only use the title topic eld that contains
very few keywords related to the topic The titleonly queries are usually short which

is a realistic snapshot of real user queries in practice
On each collection the associated topics are divided into the oddnumbered and
evennumbered topics Over those two topic subsets our proposed model is evaluated
by a fold crossvalidation In each fold one of the topic subsets is used for training
and the other subset is used for testing purposes More specically the half of the
training topics with lower topic numbers are used to train the length distribution
estimation parameters and the other half of the training topics are used to train
the score combination parameter  in Equation  Finally our proposed BML
model is evaluated by its retrieval performance on average over the two subsets of test
topics We use the TREC ocial evaluation measures in our experiments namely the
statMAP on ClueWeb B in Voorhees and Buckland  and the Mean Average
Precision MAP on the other three collections in Voorhees et al
The baseline of our evaluation is the classical BM model with dierent settings
of its parameter b By varying the b value we investigate to which extent BML
improves the retrieval performance In particular we compare the retrieval performance
of BML to BM with b   that is BM without tf normalization and
BM with its parameter b optimized All statistical tests are based on Wilcoxon
Matchedpairs Signedrank test

Comparison with BM
Tables  and  compare the retrieval performance of BML using data transform
technique to the original BM without tf normalization ie when b   and with
the tf normalization with its parameter b optimized respectively
Table  Evaluation results over the BM baseline with b   A star indicates a
statistically signicant improvement over the baseline
Coll BM Normal LogGamma LogIGD BoxCoxIGD BoxCoxGEV Lognormal Weibull
disk
WTG
GOV
ClueWeb B
Table  Evaluation results over the BM baseline with optimized setting of b A star
indicates a statistically signicant improvement over this baseline
Coll BM Normal LogGamma LogIGD BoxCoxIGD BoxCoxGEV Lognormal Weibull
disk
WTG
GOV
ClueWeb B
From Tables  and  we see that modeling document length distribution using
GEV distribution leads to the most stable retrieval performance of our proposed
lengthbased BML model This is not of a surprise as we have shown that the
GEV density ts the best to the actual document length distribution Using the
GEV density tting of the document length BML appears to outperform the
BM baseline and the improvement is statistically signicant in most cases on all

four test collections except WTG
The use of other distribution functions in particular Gamma distribution also
leads to retrieval performance over the BM baseline on some of the test collections
However their retrieval performance does not appear to be as robust as that
obtained by GEV distribution An extreme case is Normal distribution which does
not improve the BM baseline signicantly on disk WTG and GOV wether
or not the parameter b is optimized Out of the four test collections used ClueWeb B
has the most incomplete relevance assessments for which only the top documents
returned by the TREC participating runs are judged by human assessors see Buckley
and Robertson  As the top ranked documents are mostly overlong the
biase towards long documents in the document ranking could be so evident that
the length distribution of relevant and nonrelevant documents ts very well with
the distribution functions on both training and testing topics As a consequence
BML leads to extremely high retrieval performance on ClueWeb B at most cases
Tables  also show that retrieval performance of BML using mixture model
to the original BM improved
Table  Evaluation results over the BM baseline on disak
Coll BM Mixture
BM with b
BM with optimal b

Disk WTG GOV ClueWeb B








BM with b
Normal
LogGamma
LogIGD
BoxCoxIGD
BoxCoxGEV
Lognormal
Weibull
Figure  Performance of BML over BM
To visualize the improvement brought the proposed lengthbased BML model
we plot the results in Figures  for the comparison to BM with b   and
with b optimized respectively As we can see on the WTG collection although
the improvement is not as much as that obtained on other three test collections
using all six distributions the increase in retrieval performance is the evidence of
length eect in information retrieval Using Weibull distribution and normalization
transformation has the best results this may due to that Weibull distribution does
retain the skewness of data between zero and one scale on all collections very well as
we can see from Figure
By comparing the performance improvement by BML over BM with b
with the improvement over BM with the optimized b we can see that the improve

ment over the optimized b is overall of a less scale than that over BM without tf
normalization This is because optimizing the parameter b in BM has exaggerated
the length impact on the relevance weighting of term frequency tf  and in return it
reduces the impact of length relevance weighting itself on improving the document
ranking
When comparing BML with the best known results for the WTG BMLs
best MAP is  and the best published MAP is  A possible explanation of
the relatively minor improvement is as follows the data transformation on WTG
does not show much dierence between the length distribution of relevance and
nonrelevance documents Compared to largescale collections such as GOV and
ClueWeb B it leaves little room for the BML model to further improve the retrieval
performance by utilizing such dierence in document length distribution In other
words the TREC pools are biased by the length distribution Such bias is minor on
WTG and becomes evident on heterogeneous collections like GOV and ClueWeb
B which is captured by BML to boost the ranking eectiveness For ClueWeb
B we believe the best statMAP in the TREC  Relevance Feedback track ie
is achieved by combining BM with relevance feedback reported by Ye
et al  although the overview paper is not available Our model BML gives
an MAP of  For GOV on top of the retrieval baselines eg BM and

language model the best run in TREC  further improved the eectiveness using
pseudo relevance feedback and term dependency by Li and Yan  Metzler et al
Since our model only considers document length the best MAP presented
in this paper ie  is not directly comparable to the best known MAP of
For disk there hasnt been known best result for all  topics used in
the TREC  adhoc tasks According to evaluatirorg the best known MAP on
each task is   and  respectively with an average of  Note
that the above best known results are achieved by stacking additional techniques
such as relevance feedback over the retrieval baseline Therefore the results are not
directly comparable Even though BML provides an MAP of
Impact of Parameters
Experimental results in the previous section shows that on one hand BML leads
to more improvement over BM when tf normalization is disabled This is expected
since there is no length information added to BM with b   when compare to
BML On the other hand BML provides higher MAPstatMAP values when
the tf normalization parameter b is optimized From this observation a question
arises what is the impact of the setting of b on BMLs eectiveness To answer
this question Figure  plots the MAPstatMAP obtained by BML using the

dierent statistics of the document length distribution against dierent b values
from  to  BML and the original BMs retrieval performance is seen to be
correlated A better setting of BMs b leads to a better retrieval performance of
BML The document length itself do have a power as a standalone factor on the
document relevance weighting other than normalization adjustment The results for
the full range of b are illustrated in Figure  for all  test collections
Another important factor that could heavily aect BMLs retrieval performance
is the parameter  in Equation  Figure  plots the MAPstatMAP obtained
by BML against  on the four collections used As we can see that length impact
on the relevance weighting increase rst as  increase then either decrease or remain

at as  increase This is no coincidence because with only one factor ie length
among other many important factors that can aect document relevance weighting
the improvement would be limited
Another parameter in Equation  is k BML outperforms BM with
dierent settings of k although the latters retrieval performance is fairly sensitive
to ks setting The related experiments are not included in this paper for brevity
since changing ks setting does not aect the conclusions
In summary we have shown that the length information can be used for leveraging
the bias towards long documents in the document ranking The retrieval per









BM
LogIGD
Normal
Lognormal
Weibull
LogGamma
BoxCoxIGD
BoxCoxGEV
a disk










BM
BoxCoxGEV
Normal
Weibull
LogIGD
LogGamma
BoxCoxIGD
Lognormal
b GOV









BM
Weibull
LogIGD
Normal
LogGamma
Lognormal
BoxCoxGEV
BoxCoxIGD
c WTG








BM
LogIGD
BoxCoxGev
LogGamma
Normal
Weibull
Lognormal
BoxCoxIGD
d ClueWeb B
Figure  The MAPstatMAP values obtained against the parameter b
formance of the classical wellestablished BM model can be marked improved by
incorporating a lengthbased weighting component with dierent settings of BMs








LogIGD
LogGamma
BoxCoxGEV
Normal
Lognorma
BoxCoxIGD
Weibull
a disk







BoxCoxGEV
LogGamma
Lognormal
Weibull
Normal
BoxCoxIGD
LogIGD
b GOV











BoxCoxIGD
LogIGD
Lognormal
Normal
LogGamma
BoxCoxGEV
Weibull
c WTG










BoxCoxIGD
BoxCoxGEV
Lognormal
LogIGD
Normal
Weibull
LogGamma
d ClueWeb B
Figure  The MAPstatMAP values obtained against the parameter
tf normalization parameter including the optimal setting Finally we recommend
applying GEV distribution for modeling the document length distribution as it has

demonstrated eective and robust retrieval performance in our experiments In our
experiments all parameters are learned from the training data in the twofold crossvalidation
and our proposed model is trained and tested with dierent queries

Survival Approach to Diversity and Proximity
Survival Analysis
Survival analysis is a statistical methodology used for modeling and evaluating survival
data also called timetoevent data where one is interested in the occurrence
of events Cox and Oakes   Survival time refers to a variable which measures
the time from a particular starting time to a particular endpoint of interest Events
are usually referred as birth death and failure that happen to an individual in the
context of study For example in clinical trial one may be interested in the number
of days that patient can survive in the study of the eectiveness of a new treatment
for a disease Formally the survival function is dened as
St  Psurviving longer than time t
PT  t

where t is a specic time T is a random variable denoting the time of death and
P stands for probability That is the survival function gives the probability that

the time of death is later than a specied time t The survival function must be
nonincreasing Su  St if u  t and S   that is at the start of the
study the probability of surviving past time zero is one The survival function is
also assumed to approach zero as t goes to innity
We assume that the occurrence of event follows Poisson distribution with a parameter
of rate  the probability mass function is
PX  x
x e
x

where X is a random variable denoting the number of occurrences of an event
is the rate parameter denoting the expected number of occurrences The
reason of choosing Poisson distribution will be explained in the next two sections
Therefore the survival function derived from Equation  can be formally written
as
Sx  PX  x    e
Xx
i
i
i

Survival Approach to Diversity
In the context of information retrieval aspects covered by a document can be considered
as treatments a document can be considered as a patient in the clinical trial
case The number of times that an aspect has been observed can be considered as the

survival time The new information that can be provided by an aspect corresponds
to the eectiveness of a treatment
In clinical trial as the number of times that a treatment has been given to a
patient the eectiveness of the treatment to the patient decreases While in an IR
ranked list one can expect that as the number of times that an aspect has been
observed increases the new information provided by this aspect decreases This
means that the eect of an aspect to a documents novelty decreases as the number
of its occurrences increases For example in a ranked document list when aspect
stroke treatment is observed in the jth document at the rst time the information
provided by this aspect should be counted as completely new We presume that
stroke treatment in the jth document covers the topic of medications taken by
mouth for longterm stroke treatment Then when aspect stroke treatment is
observed again in the kth k  j document of the ranked list it may provide new
information about injection for shortterm stroke treatment but it is also possible
that it only provides redundant information about medications taken by mouth for
longterm stroke treatment As we can see this situation satises the properties of
the survival function described above
When an aspect is observed at the rst time this aspect can provide completely
new information This means that at time point zero the probability

of surviving is one
The probability of obtaining new information from an aspect decreases as the
number of the aspects occurrences increases This means that the probability
of surviving decreases as time increases
When the number of occurrences of an aspect is approaching innity this
aspect can not provide any new information This means that the probability
of surviving is approaching zero as time increases without bound
The occurrences of a term in a document have a stochastic element was studied
by Robertson and Walker  In this chapter we consider that the occurrences
of an aspect in retrieved documents have the same stochastic property So it is
reasonable to assume that the occurrences of an aspect follow Poisson distribution
When the aspect aj observed in the retrieved documents the probability of new
information provided by the aspect aj can be written as
Saj xj  PXj  xj    ej
Xxj
i
i
j
i

where Xj is a random variable denoting the number of the aspect aj observed in
the retrieved documents j is the rate parameter denoting the expected number of
occurrences of aspect aj  When all aspects detected at the same rates of j   j
n we name the above as single survival model In real world the distributions

of dierent aspects occurrences may be dierent from each other Therefore instead
of assuming that all aspects frequencies follow the same distribution we assume that
each observed aspects frequency follows a specic Poisson distribution with rate j
and we referred it to multiple survival model
The Process of Reranking
In order to promote ranking diversity we propose a document reranking method
which combines the novelty and the relevance of retrieved documents at the aspect
level We ranked the rst document according to
d  arg max
di
f
X
ajAdi
Paj jQg
where di is a retrieved document and Adi is the set of aspects that can be detected
from di
For other retrieved documents the document rankings should depend on which
documents the user has already seen Suppose that we have ranked top i i
documents and now we need to decide which document should be ranked at the
ith position in the ranking list The document which can deliver the newest and
most relevant aspects should be considered as the ith document in the ranking list
Therefore given previous ranked i   documents we rank the ith document using

the following scoring function
scoredi d  di  PNew and Reljdi
We assume that aspect novelty and aspect queryrelevance are independent of
each other Moreover since document di can be presented by the aspects detected
from di Equation  thus can be written as
scoredi d  di
X
ajAdi
PNew and Reljaj

X
ajAdi
PNewjajPReljaj

X
ajAdi
PNewjaj
Paj jRel
Paj

where aj is an aspect detected from document di which follows Poisson distribution
with an estimated rate parameter PNew and Reljaj denotes the probability that
aj is queryrelevant and can provide new information as well
PNewjaj in Equation  states the probability of obtaining new information
from aspect aj  which can be calculated using the survival models proposed in
Section  Since we do not usually have relevance information Paj jRel is unavailable
One possible solution as introduced in Lavrenko and Croft  is to
consider that the best bet by relating the probability of aspect aj to the conditional
probability of observing aj given the query Paj jRel  Paj jQ Thus we can

use the ranking scores of aspects from Cao et al  Zhu et al  This
twostage model that combines a relevance model and a cooccurrence model is used
for ranking detected aspects More formally the twostage model is dened as
PaijQ
X
j
Pai dj jQ
X
j
Pdj jQPaijdj Q
where Pdj jQ is the relevance model presenting whether a retrieved document
djj    N where N is the number of retrieved documents is relevant to
the query Q Paijdj Q is the cooccurrence model presenting whether an aspect ai
is associated with the query More details can be found at Yin et al
Experiment Settings and Evaluation Measures
In order to evaluate the proposed approach for promoting ranking diversity in biomedical
information retrieval we use the TREC  and  Genomics track fulltext
collection as the test corpus It is a fulltext biomedical corpus consisting of
documents from  genomicsrelated journals indexed by MEDLINE introduced
by Hersh et al  b  ocial topics from the  Genomics track
and  ocial topics from the  Genomics track are used as queries Topics are in
the form of questions asking for lists of specic entities that cover dierent portions
of full answers to the topicsGenomics collections only present a fraction of millions
of biomedical literatures indexed by MEDLINE However to the best of our knowl

edge they are the largest and the only biomedical text collections with both manual
relevance assessments and diversity evaluation available for biomedical text retrieval
research so far
There were three levels of retrieval performance that were measured in the TREC
and  Genomics tracks passage retrieval aspect retrieval and document retrieval
Each was measured by some variants of mean average precision MAP
Passage MAP Passage MAP Aspect MAP and Document MAP were four evaluation
measures corresponding to the three levels of retrieval performance introduced
in Hersh et al  b In this paper we mainly focus on aspect level and
passage level retrieval performance since our objective is to promote diversity in the
ranked list of retrieved passages Moreover aspect retrieval and passage retrieval
were also the major tasks of these two Genomics tracks
Information Retrieval Baseline Runs
For the s topics three IR baseline runs are used NLMinter DemnerFushman
et al  and MuMshFd Stokes et al  were two of the most competitive
IR runs submitted to the TREC  Genomics track NLMinter developed by
the US National Library of Medicine achieved the best performance in the TREC
Passage MAP was dened in the TREC  Genomics track which is an alternative measure
to the Passage MAP dened in the TREC  Genomics track

Genomics track in terms of Aspect MAP Passage MAP and Document MAP
reported in DemnerFushman et al  Hersh et al  It merged the retrieval
results obtained by Essie Ide et al  Indri Terrier Ounis et al a
Theme DemnerFushman and Lin  and EasyIR Gobeill et al  and employed
a humaninvolved relevance feedback method MuMshFd was developed by
the National ICT Australia Victoria Research Laboratory and also achieved top
ranked performance in the  Genomics track Stokes et al  employed
ontologybased MeSH and Entrez Gene query expansion and entitybased relevance
feedback for genomics search Another IR baseline run is an Okapi run which
is solely based on the probabilistic weighting model BM proposed by Hancock
Beaulieu et al  The performance of the Okapi run is also above average
among all results reported in the TREC  Genomics track by Hersh et al
For s topics we test our approach on three Okapi runs since other retrieval
results submitted to the TREC  Genomics track are not available In order to
nd out wether the proposed methods can work well on strong baselines as well as on
average and weak baselines we Yin et al  set dierent values to BM parameters
to obtain dierent baselines The performance of the baseline run Okapib is
also among the top performances reported in the TREC  Genomics track Hersh
httpwwwlemurprojectorgindri

et al b
Since relevance scores from dierent baselines dier in range and relevance scores
and novelty scores also dier in rangein our experiments we use the  normalization
method to normalize the scores
The best and mean results reported in the  and  Genomics tracks are
shown in Table  The performance of baseline runs are shown in Table
Table  The best and mean results in the Genomics tracks
Best MAP Mean MAP
MAP
Aspect
Passage
Passage
Document
Experimental Results on Genomics collections
Evaluation results of the proposed approach for document reranking are shown in
Table  where SingleSM and MultipleSM denote the single survival model
and the multiple survival model respectively The values in the parentheses are the
relative rates of improvement over the original results and  denotes the improvement
over the baseline is statistically signicant Wilcoxon test at the  signicance
level As we can see our approach achieves promising performance improvements

Table  Reranking Performance based on Aspect Detection Using Wikipedia
on s topics
MAP Aspect Passage Passage Document
NLMinter
SingleSM

MultipleSM

MuMshFd
SingleSM

MultipleSM

Okapi
SingleSM

MultipleSM

on s topics
MAP Aspect Passage Passage Document
Okapia
SingleSM

MultipleSM

Okapib
SingleSM

MultipleSM

Okapic
SingleSM

MultipleSM


in most cases It is worth mentioning that our approach can further improve the best
result NLMinter reported in the TREC  Genomics track by achieving
improvement on Aspect MAP and  improvement on Passage MAP
Experimental results also show that the multiple survival model slightly outperforms
the single survival model In our experiments we use the Maximum Likelihood
Estimation MLE to estimate the Poisson parameter  then calculate the Paj according
to Poisson distribution We do not manually set the rate parameters As
described in Section  the multiple survival model estimates the distribution rate
for each detected aspect while the single survival model only estimates one distribution
rate for all aspects as it assumes that all aspects follow the same distribution
Thus it is not surprised that the multiple survival model outperforms the single
survival model
We also note that in terms of Aspect MAP the improvements on the s
topics are more signicant than the improvements on the s topics This might
be due to that the average number of distinct aspects of each s topic
aspects per topic is much larger than that of each s topic  aspects per
topic descried by Hersh et al  b A topic with more distinct aspects
indicates the information need of this topic could be more diverse In this case our
approach performs better

In biomedical IR the use of domainspecic thesauri is still the most commonly
used method of integrating external knowledge Therefore it is worthwhile to compare
the reranking performance based on aspect detection using Wikipedia and
using domainspecic thesauri Table  presents reranking results based on aspect
detection using the largest thesaurus UMLS in the biomedical domain as the
knowledge resource When the UMLS is used for aspect detection performance improvements
can be obtained in terms of Aspect MAP and Passage MAP However
Passage MAP and Document MAP may decrease on some baselines From experimental
results shown in Table  and Table  we can nd that compared with
aspect detection using the UMLS aspect detection based on Wikipedia can achieve
more evident and more stable performance improvements This is because the enriched
entity pages in Wikipedia could result in a better mapping between terms in
biomedical text and concepts Moreover instead of only providing hierarchical relationships
synonyms hypernyms hyponyms among biomedical concepts like the
UMLS plenty of Wikipedia links and anchor texts can also provide more natural
relationships among Wikipedia concepts
httpwwwnlmnihgovpubsfactsheetsumlshtml

Table  Reranking Performance based on Aspect Detection Using UMLS
on s topics
MAP Aspect Passage Passage Document
NLMinter
SingleSM

MultipleSM

MuMshFd
SingleSM

MultipleSM

Okapi
SingleSM

MultipleSM

on s topics
MAP Aspect Passage Passage Document
Okapia
SingleSM

MultipleSM

Okapib
SingleSM

MultipleSM

Okapic
SingleSM

MultipleSM


Impact of the Survival Models
In order to investigate the eect of the survival models on promoting ranking diversity
we substitute the survival model with a binary novelty measuring method The
binary method measures the novelty of an aspect as follows if an aspect is observed at
the rst time the novelty score of the aspect is  otherwise  The reranking results
of using these two methods are shown in Figure  and Figure  where Multi
SM denotes the system using the multiple survival model with aspect ltering for
reranking Binary denotes the system using the binary novelty measuring method
for reranking and No Filter denotes the system using the multiple survival model
without aspect ltering for reranking The binary method only involves ltered
aspects It reranks retrieved passages using Equation  where PNewjaj
when aj appears the rst time otherwise PNewjaj   We can see that the IR
runs using the survival model outperform those using the binary novelty measure
Statistically signicant improvement on Aspect MAP can be observed using the survival
model for reranking while the binary reranking method could only achieve
signicant improvements on Aspect MAP over the baseline NLMinterWilcoxon test
at the  signicance level The performance dierences between the use of two
methods are more evident on Aspect MAP while the dierences on Passage MAP
Passage MAP and Document MAP are minor This observation indicates that us

ing the survival model has a substantial impact on promoting diversity of the ranked
list The survival model successfully measures the probability of obtaining novel
information from an aspect and makes a positive contribution to retrieval result
diversication
NLM MuM Oka NLM MuM Oka NLM MuM Oka NLM MuM Oka







Aspect Passage Passage Document
MAP Values
NLMinter
NLMinterMultiSM
NLMinterBinary
NLMinterNoFilter
MuMshFd
MuMshFdMultiSM
MuMshFdBinary
MuMshFdNoFilter
Okapi
OkapiMultiSM
OkapiBinary
OkapiNoFilter
Figure  Eects of Survival Model and Aspect Filtering on s topicsThe xaxis
presents the evaluation measures where NLM MuM and Oka in the left
gure stand for three baselines corresponding to NLMinter MuMshFd and Okapi
Experiment Results on Web Track
Experimental results on the TREC Genomics collections demonstrate that the proposed
approach is eective in promoting ranking diversity for biomedical text retrieval
However promoting ranking diversity is not only a research topic in biomed

a b c a b c a b c a b c










Aspect Passage Passage Document
MAP Values
Okapia
OkapiaMultiSM
OkapiaBinary
OkapiaNoFilter
Okapib
OkapibMultiSM
OkapibBinary
OkapibNoFilter
Okapic
OkapicMultiSM
OkapicBinary
OkapicNoFilter
Figure  Eects of Survival Model and Aspect Filtering on s topicsThe
xaxis presents the evaluation measures where a b and c in the right
gure stand for three baselines corresponding to Okapia Okapib and Okapic
ical IR but also a research topic in many other retrieval environments eg adhoc
retrieval and Web retrieval In this section we further conduct a series of experiments
on the ClueWebTB collection also known as the Category B collection
in the TREC  to evaluate our approach in the Web environment ClueWeb
TB consists of about  million Englishlanguage Web pages and was used as the
test collection in the TREC  Web track Clarke et al b
We mainly focus on the diversity task of the TREC  Web track since the
goal of the diversity task is to return a ranked list of Web pages that together provide
complete coverage for a query while avoiding excessive redundancy in the result list

in Clarke et al b The evaluation measures of the diversity task are nDCG
with    in the Clarke et al b b and an intentaware version of
precision IAP in Agrawal et al b Both evaluation measures re
ect the
diversity of a ranked list as well as the queryrelevance of retrieved documents
topics we only use the query eld of the topics from the  Web track are
used as queries in our experiments
Table  Reranking Performance on the Diversity Task of TREC  Web Track
nDCG nDCG nDCG IAP IAP IAP
BM
SingleSM

MultipleSM

BMRF
SingleSM

MultipleSM

We compare our approach with two baseline runs The rst baseline is solely
based on the probabilistic weighting model BM The second baseline is much
stronger which employs BM for document ranking and KullbackLeibler Divergence
for relevance feedback RF based on  human labeled documents reported
by Carpineto et al  Ye et al  Experimental results are shown in
Table  As we can see the proposed approach achieves signicant performance

improvements over the BM baseline in terms of nDCG nDCG
nDCG IAP and IAP The multiple survival model still consistently
outperforms the single survival model Moreover our approach can further improve
the performance of the baseline with relevance feedback in terms of nDCG This
indicates that based on this strong baseline our approach can still rank more diverse
and relevant documents to the top of the ranked list Overall the evaluation results
on the ClueWebTB collection demonstrate the eectiveness of our approach in
promoting diversity of Web search result
Survival Approach to Proximity
In literature term proximity is interpreted as query term cooccurrences or phrases
Follow the cooccurrence interpretation of the term proximity information in Croft
et al  the proximity among query terms is interpreted as ngram frequencies
In the context of modeling term proximity information the importance of the ngram
in the document is measured by the cooccurrence probability The main focus of
this section is modeling cooccurrence probability of ngram among query terms

Reranking Model Based on Term Proximity
The probabilistic document ranking PDR for a given query using the term proximity
evidence pi is given by van Rijsbergen  as follows
PDR rank
X
in
i  Scored pi
where Scored pi is the ith ngram relevance score Dierent ngram models are
combined through a linear function with a parameter i to adjust the relative importance
of each ngram modelThus a major diculty is how to assign the relevance
score using term proximity between more than one terms ie when i is larger than
or equal to
There could be many ways to assign the proximitybased relevance scores such
as those mentioned in Section  Here we adopt the cooccurrence interpretation
of term proximity introduced by van Rijsbergen  where the proximity among
query terms is represented by the ngram frequencies then the ngram relevance
score of a document d for a query Q is given as follows
Scored Q pn
X
tttnQ
w kn  tfn
K  tfn
qtwn
where an ngram consists of n query terms t t  and tn In the context of the
ngram model the above variables can be interpreted as follows

tfn is the ngram frequency A simple method for computing the ngram frequency
is to count the number of cooccurrences of query terms A more
sophisticated solution is to consider information provided by ngram frequency
a decreasing function of the distance between query terms We propose various
novel methods for computing the ngram frequency which is described in
details in the next section
w is the raw term weight following the rewritten point formula proposed
by Robertson et al  is given as
w  log
N  Nn
Nn

where Nn is the ngram document frequency  ie the number of documents in
which the ngram occurs for at least once This Equation is corresponding to
the w in Equation
qtwn is the ngram weight it is given by the average query term weight of the
ngram terms t t  and tn qtwn
P
in qtwti
n
K is given by K  kn

bn  bn
ln
avg ln

where the variables are dened
in the context of the ngram model The length ln refers to the number of
windows in a document which is given by l  n   avg ln is the average
number of windows in a document given by
P
dColl ld  N  n

where

P
dColl ld is the number of tokens in the whole collection
Each individual bi of ngram model is optimized by Simulated Annealing proposed
by Kirkpatrick et al  on training queries while only the ngram
model is enabled This is equivalent to setting the linear combination weight
i see Equation  of other ngram models to
The other parameters kn and kn are the same parameters for each ngram
model as those in the contentbased BM in Equation  In experiment
we default them to k   and k   for every ngram model to reduce
the number of parameters to be optimized
Term Proximity Modeling
The simple way of modeling term proximity is the windowbased ngram frequency
counting NC method which has been popular in previous studies on using term
proximity for IR eg Metzler and Croft b Plachouras and Ounis b
The basic idea of the windowbased ngram counting method is to segment the
document into a list of sliding windows with each window having a xed window
size wSize If a document has a length of l and the window size is set to wSize
the document is then segmented into wSize sliding windows where each window
contains wSize consecutive tokens For example if a document has four tokens A

B C and D and the window size is  there are two windows in this document
namely A B C and B C D The ngram frequency is then dened as the number
of windows in which all ngram terms cooccur
However the ngram counting method does not take the actual distance between
query terms into account and any ngram terms appear together within a window
is counted as one occurrence of the ngram Another possible downside is that it is
unable to dierentiate between ngrams in which the ngram terms appear loosely
and tightly in the text An alternate solution is to approximate a proximitybased cooccurrence
probability of the ngram terms in the document d The distance between
the ngram terms is characterized by the shortest distance between any two ngram
terms in the ngram and other ngram terms appears in between For example if a
window consists of a sequence of the following  tokens Q A Q Q B Q where
Q Q and Q are the trigram terms and A B are tokens in the document the
distance between the trigram terms is  in this case which is the number of tokens
between Q and Q
Therefore the ngram frequency becomes
tfn
X
ix
PiQn i
where x is the number of windows in which the ngram terms cooccur i is the
distance between the ngram terms in the ith window in which the ngram terms

cooccur To simplify the approximation we assume that only the two ngram terms
that have the shortest distance in the window and the other ngram terms appear
in between is taken into account In the context of modeling the term proximity
information the importance of the ngram in the document is measured by the
probability of cooccurrence and its proximity denoted as S  The cooccurrence
probability corresponds to the eectiveness of a treatment or a mean of modeling
of the distance where  is the distance between Qi and Qj  More details can be
founded in He et al  Remind that Qi and Qj are the two ngram terms with
the shortest distance in the text window A text window can be interpreted as a
patient in the clinical trial case and the distance between Qi and Qj  composed of
a sequence of words can be interpreted as the survival time that is the time that
it takes for the treatment to cure the patient In other words in a text window as
the distance between the ngram terms increases the importance of the ngram in
the document decreases The ngram has the highest importance when the ngram
terms are adjacent to each other that is when the distance or survival time is zero
So the probability of the ngram cooccurrence and proximity can be approximated
by utilizing survival function on the Homogeneous Poisson Process SurvHPP exponential
distribution SurvExp and empirical distribution SurvEmp as follows

T follows Poisson distribution
S     F     e
X
i
i
i

T follows exponential distribution
S   PT     e
The empirical survivor function is
S

n
Xn
i
I
where I  is an indicator function
Experimental Setup
We evaluate our proposed methods on adhoc topics over four largescale TREC
Web collections These four collections are the most recent Web collections with
associated adhoc test topics which are currently the best snapshot of the real Web
environment The WTG GOV and ClueWebB collections are introduced in
the Subsection  The Blog collection includes  blog feeds collected
over an  week period from December  to February  Following the ocial
TREC setting in Ounis et al b we index only the permalinks which are the
Other TREC Web collections are not used in that they do not have corresponding adhoc test
topics

blog posts and their associated comments For all four test collections used each
term is stemmed using Porters English stemmer and standard English stopwords
are removed
Each topic contains three topic elds namely title description and narrative
We only use the title topic eld that contains few keywords related to the topic
The titleonly queries are usually short which is a realistic snapshot of real user
queries in practice On each collection we evaluate our proposed model by a twofold
crossvalidation The test topics associated to each collection are split into two
equal subsets namely the oddnumbered and evennumbered topics In each fold we
use one subset of topics for training and use the remaining subset for testing The
overall retrieval performance is averaged over the two test subsets of topics We use
the TREC ocial evaluation measures in our experiments namely the Mean Average
Precision MAP on WTG and GOV in Voorhees et al  the topical MAP
on Blog in Ounis et al b and the statMAP on ClueWeb B
Experimental Results
In this section we evaluate our proposed proximitybased model in a bigram setting
rst The evaluation baseline is the original unigram BM model in Equation
Information about the ClueWeb dataset can be found at the following URL
httpbostonlticscmueduDataclueweb

proposed by Robertson et al   Compared to this baseline we evaluate our
proposed model at two dierent levels First we compare the performance of the
bigram BM model with the unigram BM baseline Second we compare the
unigram BM model alone with the combination of the unigram and bigram BM
models
Table  The MAPstatMAP values obtained by the unigram BM baseline and
the bigram model
Unigram Bigram unordered
Coll BM NC SurvHPP SurvExp SurvEmp
WTG
Blog
GOV
ClueWeb B
Unigram Bigram ordered
WTG
Blog
GOV
ClueWeb B
Table  compares the retrieval performance of the bigram models with the
unigram BM model The two variants namely the ordered and the unordered
bigram models are also tested Results in Table  show that the bigram BM
model is not as good as the unigram model This observation is in line with the
ndings in previous studies eg Metzler and Croft b We suggest that the

relatively low performance of the bigram model could be due to the fact that the
bigram model is unable to properly measure the informativeness of the individual
query terms that do not form a concept with other query terms Even though the
bigram model still provides a descent retrieval performance which has the potential
to lead to improved retrieval performance by combining the unigram model with the
bigram model
Table  The MAP values obtained by the unigram BM baseline and the bigram
model on WTG
Model Unigram Unordered Ordered
NC
SurvHPP
SurvExp
SurvEmp
Table  The MAP values obtained by the unigram BM baseline and the bigram
model on Blog
Model Unigram Unordered Ordered
NC
SurvHPP
SurvExp
SurvEmp
Next we examine the retrieval performance of the linear combination of the
unigram and bigram models The related experimental results on the four collections

Table  The MAP values obtained by the unigram BM baseline and the bigram
model on GOV
Model Unigram Unordered Ordered
NC
SurvHPP
SurvExp
SurvEmp
Table  The statMAP values obtained by the unigram BM baseline and the
bigram model on ClueWeb B
Model Unigram Unordered Ordered
NC
SurvHPP
SurvExp
SurvEmp
used are listed in Tables    respectively From these tables we observe that
the combined model outperforms the unigram BM model in most cases To be
more specic the NGram Counting NC method and the Survival Analysis over
exponential distribution SurvExp result in statistically signicant improvement
over the unigram BM baseline on three out of four test collections used We also
observe insignicant improvement brought by BMP on ClueWeb B Furthermore
the unordered combined model is consistently better than the ordered one This
indicates that the order of appearances of query terms is not important for adhoc

retrieval on the collections used In addition to the bigram models tested in the
Table  The MAPstatMAP values obtained for evaluating the trigram model
A star indicates a signicant improvement over the unigram BM baseline
Coll BM NC SurvHPP SurvExp SurvEmp
Unigram UnigramBigram unordered
WTG
Blog
GOV
ClueWeb B
Unigram UnigramTrigram unordered
WTG
Blog
GOV
ClueWeb B
Unigram UnigramBigramTrigram unordered
WTG
Blog
GOV
ClueWeb B
previous section we evaluate the eectiveness of our proposed model in trigram
setting in this section We do not experiment beyond the trigram model since
only few titleonly TREC test queries contain more than four nonstopword unique
keywords
Table  presents the results for evaluating the trigram model The combination
of the unigram model with either the bigram or the trigram model can

provide a statistically signicant improvement over the unigram baseline in many
cases However the improvement brought by the trigram model is not as high as
those brought by the bigram model This might be due to the fact that the bigram
model better utilizes the latent association between query words than the trigram
model Moreover a combination of the unigram bigram and trigram models performs
better than the combination of the unigram and bigram models while the
improvements are rather marginal on all four collections We only report the results
with the unordered models in that the ordered models lead to similar conclusions
while their retrieval performance is slightly lower In summary the trigram model is
shown to be a useful addition to the bigram model although the combination of the
unigram and bigram models are sucient to provide reliable retrieval performance
on the collections used

Conclusion and Future Work in IR
We have demonstrated the eectiveness of incorporating document length aspect and
term proximity information into the classical probabilistic IR models In Chapter
we study the impact of document length on its relevance based on the assumption
that a document may exhibit both Verbosity and Scope hypotheses and show the
evidence to its retrieval In Chapter  we apply the survival function to estimate
the novel information provided by aspects to promoting the diversity in the ranked
document lists and term proximity to improve retrieval performance
Based on the current work we plan to extend our work to the other IR models
such as language model and PL possibly by incorporating with machine learning
methods
Document length has been empirically proved as an useful factor in document
retrieval There is a need of nding a simple way to model the impact of
document length on document relevance when document style and structure
change dynamically nowadays

In fact whether a repeated aspect could provide novel information may depends
on the context it appears We thus plan to take aspects contextual information
into account in our future work We also plan to adapt our approach to other
domains such as Web retrieval
Term proximity that can be employed not only by BM in text retrieval we
also consider to extend the application of our proposed model to retrieval from
structured documents such as the XML retrieval An XML document is an
ordered and labeled tree where the nodes are usually composed of short text
and tokens of the same word appear at dierent levels in the tree structure
This poses a challenge for the application of our proposed model to the XML
documents since it is dicult to dene the notions of ngrams and window size
in such a tree structure
From next Chapter we will start on the topic of asymptotic methods and its
applications

Literature Survey on Asymptotic Methods
Fisher  dened the likelihood function as a function of the parameters of
a statistical model which is proportional to the joint density of the model In
Fisher extended the studies and proposed the method of maximum likelihood
estimate MLE as a mean of a systematic way of estimating the parameters based
on the observed sample Neyman and Pearson  proved that the likelihood
ratio test is the most powerful test for any given size
test Under regularity
conditions as stated in Wilks  he proved that the limiting distribution of
the likelihood ratio is a  distribution Wald  extended Wilks work on the
distribution of the likelihood ratio statistic to a more general situation and derived
the limiting distribution of the maximum likelihood estimate Additionally Rao
introduced score test which is also based on the likelihood function These
likelihood inference methods have rates of convergence On and is referred to
as the rstorder approximation methods Even though the rstorder methods are
widely used they generally do not give accurate approximation especially when the

sample size is small In recent literature Fraser and Reid  derived a more
accurate likelihoodbased asymptotic method with rate of convergence On
This section reviews some key concepts and denitions in asymptotic approximation
inference In Section  we review the likelihood function and maximum
likelihood estimate The standard rstorder asymptotic techniques are also examined
In Section  the saddlepoint approximation to the density of the mean of
n independent identically distributed random variables is introduced We give the
brief presentation about the thirdorder asymptotic method in Section
Standard Firstorder Likelihoodbased Asymptotic Meth
ods
Assume a statistical model has probability density function or probability function
f   where         k   is a kdimensional parameter The likelihood
function dened by Fisher  for any random sample y  y y     yn
obtained from the above model is
L  L y     yn  cfy     yn
for values of  within a given domain where c   is a multiplicative constant
and fy     yn  is the value of the joint probability distribution or the joint

probability density function of the random variables Y     Yn evaluated at Y
y     Yn  yn The corresponding loglikelihood function is
y     yn  a
Xn
i
logfyi
where a  R is an additive constant Without lost of generality a is set to be zero
hereafter in this dissertation
Let  be the overall MLE which can be obtained by solving






Let      be a scalar parameter of interest and  be the nuisance parameter with
size k   denoted as      Now consider the null hypothesis
H
is the constrained MLE of  which maximize  for a given      Generally
when  is explicitly known   is obtained by solving






However when  is not explicitly available   can be obtained by applying the
Lagrange multiplier technique for maximizing  subject to the constraint
Since the closed form of  and   are not always available numerical methods
are often required The tilted loglikelihood function is dened as


with maximum value at   and  is the Lagrange multiplier which maximized the
Note that this tilted likelihood has the property      for a given
constraint
Moreover throughout this dissertation with    the following regularity conditions
are assumed to hold
fy    is twice continuously dierentiable in a neighborhood of

Z
sup
N
jfy jdy   and
Z
sup
N
jfY  jdy
where fy
fy

and fy
fy

Ey
y  exists and is nonsingular where




Z
sup
N
jy jdy   where


Under these regularity conditions applying Central Limit Theorem and Taylor
expansion to  the following can be obtained
d  Nk var or equivalently
fvarg  d
k
where var  i  E j is the Fisher expected full information
matrix and j   is the observed information matrix
d  Nk var or equivalently   var   d
k
where var  i



h

i
d
k
Note that although i could be dicult to obtain it can be approximated by
j When we are interested in the inference for   the following three test
statistics are usually used
Signed loglikelihood ratio statistic
r  r   sgn      f   g


Wald statistic
q  q
q
var

Score statistic
S  S
q
var

where
var      j
















Engle  showed that the three tests are asymptotically equivalent
The pvalue functions based on   and  are dened as
p


r
q
S

where  is the cumulative distribution function of N These methods all have
rate of convergence On and are generally referred to as the rstorder methods
Hence a central
condence interval for   is

minfp
p
g maxfp
p
g


The rst two statistics are more popular in terms of application For nite sample
Doganaksoy and Schmee  found that generally  has better coverage
properties than  As we have discussed before Neyman and Pearson
proved that  gives the most powerful test However they also pointed out the
is more popular in applied analysis than  because of its simplicity in application
For example a
condence interval for   is approximately
z

q
var
Although the three rstorder methods are widely used in hypothesis testing they
do not perform well when the sample size is small or when the underlying distribution

is far away from the normal distribution In the next two subsections higherorder
asymptotic methods will be introduced
Saddlepoint Approximation
Daniels introduced saddlepoint approximation to statistics in  This method
approximate the density of the mean of n independent and identically distributed
iid random variables and it is very accurate but could be very complicated in
terms of computation
Assume Y     Yn are iid random vectors with size k from a model with density
fY   The moment generating function is dened as
Mt  Eet

Y
and cumulant generating function is
Kt  logMt
The saddlepoint approximation for the density of the mean of n independent
identically distributed random variables ie  Y  nPn
i Yi given by Daniels
is
f Y y  k

n
Kttt


exp

nfKt  tyg

f  Ong

where
Ktt
Kt
t

tt
y
is the saddlepoint and
Kttt
Kt
tt

tt
t is known as the saddlepoint Note that  has a relative error of On
A statistical version of the derivation of the saddlepoint approximation can be
obtained by combing exponential tilting and Edgeworth expansion techniques and
then collect the rst four terms For detailed review of the derivation please refer
to BarndorNielsen  BarndorNielsen and Cox    and Reid
The resulting density is
f Y y  a

n
Kttt


exp

nfKt  tyg

f  Ong
where a is the normalizing constant Durbin  showed that error term of the
saddlepoint approximation  is reduced to On when k is replaced by
a normalizing constant a in
For a canonical exponential family model with density
fY y   exp fy  c  hyg
and the loglikelihood function can be written as
ny  nc

with the cumulant generating function
Kt  logMt  c  t  c
there exists a oneone transformation from  Y to
Kt   Y
where  Y is a minimal sucient statistic for  BarndorNielsen  showed
that the saddlepoint approximation of the density function of  is
f   ajjj exp
n

o
On


where a is a normalizing constant For models outside the exponential family setting
if an ancillary statistic is available there is onetoone correspondence between the
minimum sucient statistic and an ancillary statistic An example of the construction
of the ancillary statistic can be found in BarndorNielsen  or Barndor
Nielsen and Chamberlin  For a general model if the MLE is not a oneone
transformation of the minimal sucient statistic whereas  continues to provide
an approximation to a conditional density of  by conditioning on an appropriate
ancillary statistic A The same result derived by BarndorNielsen and Cox
is
fjA   aAjjj exp
n

o
On



The above approximation is accurate to On for exponential families and transformation
models Detailed discussion of the saddlepoint method and its application
in statistics can be found in BarndorNielsen     McCullagh
Fraser  Reid   and BarndorNielsen and Cox
However there is no systematic method to construct ancillary statistic for a general
model Moreover ancillary statistic may not exist and even if it exists it may not
be unique
The Thirdorder Asymptotic Methods
Intensive research on higherorder asymptotics was intrigued by BarndorNielsen
and Cox  which highlighted the usefulness and accuracy of the saddlepoint
approximation to the density function of  Y  the mean of n iid random variables as
well as the cumulative distribution function for Y  F Y y
R y
f Y sds However
this approximation may not have closed form and numerically integration is required
which will introduce signicant error when analytical solution does not exist
Following the idea of saddlepoint approximation Lugannani and Rice  calculated
the cumulative distribution function by inverting the characteristic function
Fourier transformation
F Y y  P  Y  y  r  r


r


q

On

where r  sgnt
q
nty  Kt and q  t
q
nKttt Note that t is the saddlepoint
dened in Section  which satised Ktt  y
For the canonical exponential family dened by  the likelihood formulation
of the statistics r and q is
r  sgnq
h
f  g
i

q    fjg


which coincide with the signed likelihood ratio statistic and the maximum likelihood
departure as dened by  and  for a scalar parameter of interest situation
Hence the pvalue function of  approximated by Lugannani and Rice  formula
is
p  r  r


r


q


BarndorNielsen  derived alternative approximation that incorporates the
correction term into the quantile of the normal cumulative distribution
F Y y  P  Y  y  r

On


where
r  r

r
log
q
r


and showed that it is asymptotically distributed as a standard normal distribution
with a relative error of On r is referred as modied signed loglikelihood ratio
statistic Hence the approximated pvalue function of
p  r
It is interesting to note that the BarndorNielsens method adjusts the signed
loglikelihood ratio statistic such that the pvalue function obtained from r is close
to the true pvalue function whereas the Lugannani  Rice method adjusted the
pvalue function obtained from the signed loglikelihood ratio statistic such that it
is close to the true pvalue function Fraser  and Jensen  showed that
these two adjustments are equivalent up to thirdorder accuracy
Equations  and  are generally referred as the thirdorder methods
For exponential family models and transformation models  and  can be
obtained easily However for general models q could be dicult or impossible to
derived since it is based on the existence of ancillary statistic In practice there
is no accessible procedure available for the construction of an ancillary in a general
context In Chapter  a route to the thirdorder likelihood inference for any general
statistical model will be discussed

ThirdOrder Likelihood Inference for a
General Statistical Model
This chapter details the mechanics of the likelihoodbased thirdorder methods for a
general statistical model The advantage of these two proposed thirdorder methods
Lugannani and Rice and BarndorNielsen are their prominent accuracy for even
when sample size is small and applicability on obtaining inference for any scalar parameter
of interest It only depends on the likelihood function and its rst sample
space derivative at the data points However calculating the pvalues may encountered
singularity at the maximum likelihood value and be numerically instable in
the neighborhood of the maximum likelihood value Fraser et al  proposed
a bridging method to deal with the instability and singularity problem when the
MLE is close to the hypothesized value therefore it will not be discussed in detail in
this dissertation When the parameter of interest   cannot be expressed explicitly
computation problems arise for any likelihoodbased methods For example the

considered in this dissertation is an integral which has no known closedform The
numerical diculties will be discussed in the next chapter and a penalized likelihood
method is proposed to deal with the constrained maximization problem for the model
we considered will also be presented in the next chapter
The organization of the rest chapter is as follows Section  studies the canonical
exponential family model with the presence of the explicitly known nuisance
parameter  and then extended to a general exponential model in Section  An
illustrated example is also given in this section via BehrensFisher problem A general
algorithm to the thirdorder likelihood inference for any statistical model will be
introduced in Section
Canonical Exponential Model
Consider a canonical exponential family model
fy   exp fy  c  hyg
where      and our parameter of interest is   It is easy to see that y
is a sucient statistic and  is the canonical parameter for the model For any
random sample from above model the signed loglikelihood ratio statistic r is remain

unchanged as in  and it is
r  r   sgn
n

h

io

where   is the log likelihood evaluated at constraint MLE
Applying saddlepoint procedure Fraser et al  on the joint and on the
marginal likelihood functions the standardized maximum likelihood departure in
the canonical parameter space becomes
q  q



j


j





where j is the observed overall information matrix evaluated at the overall
MLE  and j  is the observed nuisance information matrix evaluated at the
constrained MLE   Detailed derivation of  is discussed in Fraser et al
Therefore we can apply either LugannaniRice formula  or BarndorNielsen
formula  to obtain p  with r and q dened above These two approximations
both have thirdorder accuracy
To illustrate the application of thirdorder method for canonical exponential family
model the exponential distribution the simplest case is considered Suppose
y     yn is a sample from exponential distribution with rate    and its probability
density function can be written in canonical form as
fy   exp fy  log g  y

This is also known as the Gamma  distribution Then the loglikelihood is
l  n log
Xn
i
yi
The overall MLE can be obtained easily ie
Pn n
i yi
and j  l
n

The signed loglikelihood ratio statistic and the standardized maximum likelihood
estimate departure given in  and  can be easily obtained as
r  sgn  f   g
q

n



Hence we can approximate pvalue function for  by  or
Moreover let T
Pn
i Yi where Yi is iid from exponential model above from
distribution theory T  Gamman  Hence for a given
PT  t
Z t

fT s ds
Then the exact pvalue function for  is
p    PT  t
To compare the accuracy of thirdorder methods we generated three data set
from above exponential distribution with    and sample size to be   and
The simulated data are recorded in Table



n
q
Probability
Exact
r
Wald
LR
BN
Figure  p for Data Set



n
q
Probability
Exact
r
Wald
LR
BN
Figure  p for Data Set



n
q
Probability
Exact
r
Wald
LR
BN
Figure  p for Data Set

Table  Three simulated data sets
Data Set Observations Sample Size n







Figures  to  display the pvalue functions obtained from Exact r qWald
Equation LR and Equation BN for the three data sets The horizontal
lines indicate the two nominal levels  and  for the  central condence
interval respectively Table  gives the corresponding  condence intervals
Both the plots and Table  show the outstanding performance of the thirdorder
methods over the rstorder methods and thirdorder methods have remarkable
accuracy when sample size is small

Table   central condence intervals for
Method n   n   n
Exact
r
Wald
LR
BN
General Exponential Family
Now consider a general full rank exponential family model
fy   exp fty  c  htyg
with the scalar parameter of interest being      and      with  be
nuisance parameters vector which is explicitly known Under the set up of model
the natural parameter and natural variable are  and y and  and ty
are the canonical parameter and canonical variable which are functions of the natural
parameter and natural variable respectively We restrict our attention to the
fullrank exponential family model where the canonical parameter and the natural
parameter have the same dimensionality since most commonly used distributions in

the exponential family belong to this category For the curved exponential family
model that is the dimension k of the parameter vector        k is less than
the dimension s of the vector        s detailed discussion can be
found in BarndorNielsen
The signed loglikelihood ratio statistic r  r  is invariant to reparameterization
it remains unchanged and is same as in  However the quantity q  q
has to be reexpressed in the canonical parameter  scale
For the general exponential model above we have
y  ty  c
Fraser  derived
y S  y
for the observation y with  being the corresponding MLE where
y
y
y  S
S y
y  S y
y


Let  be a rotated coordinate of  that agrees with   at   is dene as


jj  jj

which can be viewed operationally as the canonical parameter of in  scale The
is the row of
that corresponds to   jj jj is the square length of the

vector   and  is the derivatives of  with respect to  The calibrated
version  of  is basically a vector from the space spanned by the columns of
the  and its direction depends on the constrained MLE for given  Hence
j   j is a measure of departure of    from   in  scale
Since    by the chain rule in dierentiation we have the full and
nuisance information determinants recalibrated on the  scale
jjj  jjjjj and jj j  jj jj
j
where  is the derivatives of  with respect to
An estimated variance for



obtained by Fraser et al  in
scale is
cvar
jj j
jjj

Thus the standardized maximum likelihood departure of   in  scale is
q  q   sgn      j   j

jjj
jj j


Therefore p  can be obtained by Lugannani  Rice method  and Barndor
Nielsen method  with r and q being dened in  and  respectively
Thus a
condence interval for   is

min

p
p


max

p
p




As an example we consider inference for the dierence of two independent normal
means which has been widely studied in statistical literature Typically the
variances are assumed to be unknown and must be estimated When we assume
equal variances then a pooled estimate of the common variance is used and the
test statistic is exactly distributed as a Student t distribution However without
making the equality of variances assumption the problem is then the wellknown
BehrensFisher problem where no exact distribution of the test statistic is available
There exists many approximate solutions In this section we apply the thirdorder
methods discussed in the previous chapters to the BehrensFisher Problem with an
additional assumption that the ratio of the two variances is known which is examined
by Schechtman and Sherman  More discussion can be found on the
thirdorder methods to BehrensFisher Problem proposed by She et al  when
the ratio of the two variances is unknown The comparison with rstorder approximation
methods and the methods proposed by Schechtman and Sherman  is
presented
Let x  x  xn and y  y  ym be random samples from two independent
normal distribution with mean and variance x
x  and y
y respectively
Assume
x and
y are unknown but with the ratio
y
x  c known Our parameter
of interest is    x  y

The loglikelihood function can be written as
l
m  n

log
x


x
Xn
i
xi     y

c
x
Xm
j
yj  y
where     y
x
The overall MLE     y
x are
x  y y  y
x
Pn
i xi  x
c
Pm
j yj  y
m  n

Then the observed full information matrix j can be derived as

BBB
n
x
n
x
nx     y
x

n
x
n
x

m
cx
nx     y
x


my  y
cx

nx     y
x

nx     y
x


my  y
cx


m  n
x



x


X
xi     y

c
X
yj  y

CCCA

Hence the determinant of the observed information matrix evaluated at the overall
MLE is jjj
mnm  n
c
x
The constrained MLE      y
x  can also be obtained directly where
y
n
n  mc
x
mc
n  mc
y
n
n  mc


x
P
xi     y
c
P
yj  y
m  n

and y     y
Similarly the observed nuisance information matrix j is

BBB
n

x

m
c
x
nx     y

x
my  y
c
x
nx     y

x
my  y
c
x

m  n

x


x
X
xi     y

c
X
yj  y

CCCA


Therefore the determinant of constrained nuisance information matrix is
jj j
m  nm  cn
c
x
The signed loglikelihood ratio statistic  can be written as
r  sgn
vuut
m  nlog


x

x



This model is a canonical exponential family model with canonical parameter


y

x

y

x



x


Then we have


BBBBBB

x
x    y
x

x y
x

x

CCCCCCA

jj
x


BBBBBB

x    y
x

x y
x

x

CCCCCCA


x
x
x
Thus from  we have
q  q
r
mn
m  cn

x

x

The pvalue function for   can then be approximated from either by the Lugannani
Rice method  or by the BarndorNielsen method  with thirdorder
accuracy where r and q are dened in  and  respectively
Schechtman and Sherman  showed that a
condence interval
for    x  y under set up in the previous Section  is

x  y  t

sp
r

n

c
m

x  y  t

sp
r

n

c
m

where x
Pn
i xi
n  y
Pm
j yi
m  s
x
Pn
ixix
n  s
y
Pm
jyjy
m  s
p  ns
xms
yc
nm
and t is the
th percentile of the tdistribution with n  m   degrees
of freedom They claimed that their method is good in terms of size and power To
compare the accuracy of the thirdorder methods with the Wald method the signed
log likelihood ratio method and the Schechtman and Sherman  method Monte
Carlo simulation studies were conducted
We generated  simulated samples for some combinations of the parameters
For each simulated sample we calculate the  condence intervals for   obtained
by the thirdorder methods in Section  ie LR and BN with the
Wald method Wald the signed loglikelihood ratio method r and the Schechtman
and Sherman  method SS For each simulated setting we report the
proportion of   that falls outside the lower bound of the condence interval lower
error the proportion of   that falls outside the upper bound of the condence in

terval upper error the proportion of   that falls within the condence interval
central coverage and the average bias Average Bias which is dened as
Average Bias
jlower error  j  jupper error  j


The nominal values for the lower and the upper errors the central coverage and the
average bias are    and  respectively These values re
ect the desired
properties of the accuracy and symmetry of the interval estimates of
The simulation standard errors for these three quantities are   and
respectively Results are recorded in Tables  to  It is clear that the
results from Wald method and signed loglikelihood method are not satisfactory especially
when the sample sizes are small Results from the Schechtman and Sherman
method LR and BN are identical for large sample size and almost indistinguishable
for small sample sizes they are all within  simulated standard errors

Table  x      x  y
x
y  c
x n   and m
cknown Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
SS
LR
BN
Wald
r
SS
LR
BN
Wald
r
SS
LR
BN
Wald
r
SS
LR
BN

Table  x      x  y
x
y  c
x n   and m
cknown Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
SS
LR
BN
Wald
r
SS
LR
BN
Wald
r
SS
LR
BN
Wald
r
SS
LR
BN

Table  x      x  y
x
y  c
x n   and m
cknown Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
SS
LR
BN
Wald
r
SS
LR
BN
Wald
r
SS
LR
BN
Wald
r
SS
LR
BN

General Statistical Model
The key step of the thirdorder method is to nd the canonical parameter
For a general statistical model when suciency and ancillarity do not reduce the dimension
of the variable to that of the parameter an approximation of ancillary statistics
could be obtained BarndorNielsen  and McCullagh  suggested
a dierent ways of constructing approximate ancillary statistics but the methodologies
are problem dependent However ancillary statistics may not exist and even if
it exits it may not be unique Moreover the feasible methods are lacking for tail
probability approximation Fraser  developed the tangent exponential model
to approximate an asymptotic model with thirdorder accuracy for a model with
variable and parameter of the same dimension and Fraser and Reid  extended
to more general case of variable of larger dimension than parameter by constructing
an ancillary direction and using observed likelihood The tangent exponential model
can be fully characterized by its likelihood function and sample space derivative of
the likelihood function at the data point showed by Fraser and Reid  For the
one dimension case the tangent exponential model is determined by the rst row and
the rst column of the coecient matrix of the Taylor expansion of the loglikelihood
function at both the MLE and observations
The term ancillary direction is refer to the tangent direction to the ancillary

statistic surface at the observed data point It can be retrieved from an appropriate
pivotal quantity in the applications without construction of approximate ancillary
statistic Fraser and Reid
Consider realvalued parameter  and response y Assume the cumulative distribution
function of Y  Fy  has nonzero derivative with respect to  and the
variable y is monotone increasing in  Note that Fy   U  hence it is a
pivotal quantity The total dierential for F at  is
dFy   Fyy j ydy  Fy j yd
with Fyy  and Fy  being the partial derivatives of Fy  with respect to both
y and  respectively y is the observed value for y With the probability level F
held constant the ancillary direction is given by Fraser  as
V
y

F
y y Fy

y
which depicts the change in y that corresponds to the change in  Denote the
ancillary direction as V  v v     vd V records how y changes when  changes
For a pivotal quantity zy  other than the cumulative distribution function
the ancillary direction is given by
V
y

z
y y zy

y


Note that vectors v v     vd are tangent to rst derivative ancillaries for parameter
changes in linearly independent direction at    For more details on the
ancillary directions tangent to the level surface of an approximate ancillary statistic
see Fraser  Fraser and Reid
If the model is conditional model which is conditioning on some ancillary statistic
then the conditional likelihood gradient becomes the full likelihood gradient tangent
to the ancillary surface Thus Fraser et al  derive that the reparametrization
can be obtained by dierentiating the full likelihood in the directions V


V
l y

y
ly yV


Xn
i

yi
l yvi
Xn
i

yi
l yvik


This quantity is the locally dened canonical parameter for the tangent exponential
model at the data point y So the observed loglikelihood l y and the ancillary
direction V together will produce a locally dened canonical parameter  The
tangent exponential model provides the full thirdorder pvalues for the original model
Fraser et al
For a special case when the dimension of variable y is same as the dimension of
parameter  the new parameter   would be
yy
y
y

y

Fraser  and Fraser and Reid  proved that to secure thirdorder accuracy
in  and  an ancillary statistic of secondorder is sucient
Once we have the tangent exponential family model and its locally dened canonical
parameter the methodology in Section  can be directly applied to approximating
the tail probability by using either the BarndorNielsen method  or
the LugannaniRice method  Thus a      condence interval for
can be obtained accordingly The results always apply for the cases when the
nuisance parameter is explicit or implicit Computational issues encountered in obtaining
constrained MLE for likelihoodbased thirdorder method will be addressed
in Chapter  Illustration of thirdorder method for general statistical model will
be presented in Chapter  For the rest of this dissertation we only consider the
BarndorNielsen method  since we can see the results from BarndorNielsen
method  and LugannaniRice method  in Section  are almost identical
regardless the sample size

Computational Issues
Calculating constrained MLE is one of the major steps of the likelihoodbased thirdorder
method Computation problems always arise for the likelihoodbased method
because of the optimization problem especially when the constrained MLE involves
a constraint which cannot be expressed in closedform In our case the constraint
is equating the stressstrength reliability in a form of an integral This problem will
be discussed in Section  a penalized likelihood method is proposed in Section
to deal with this numerical complication of maximizing the constrained likelihood
model illustration of proposed penalized likelihood method using a real life example
will also be presented
Computation Problem
The constrained MLE   for a given   is obtained by maximizing the loglikelihood
function  subject to the constraint R       Formally this optimiza

tion problem can be approached using the Lagrange multiplier method But when
the constraint has no closed form representation the standard Lagrange multiplier
method will encounter numerical diculties In the rest of this chapter Burr type
X distribution will be used as an illustration
The Burr type X distribution was introduced by Burr  Using the notation
in Raqab and Kundu  let Y be distributed as a Burr type X distribution
BurrX  the distribution function can be written as
Fy

ey

y
where  is the shape parameter and  is the scale parameter Let x  x     xn
and y  y     ym be the random samples from BurrX  and BurrX
respectively Then the loglikelihood function is
l    x y
n log   n log
Xn
i
log xi
Xn
i
xi
Xn
i
log
h
exi
i
mlog   mlog
Xm
j
log yj
Xm
j
yj
Xm
j
log
h
eyj
i

where       and     are positive The parameter of interest

is the stressstrength reliability R which is dened as
R  R  PY  X
Z

Z x

fx   fy   dx dy

Z


x ex

ex
h
ex
i
dx
The constrained MLE        for a given   is obtained by maximizing
the loglikelihood function in  subject to the constraint R       in
In this case the constraint on  values is expressed by equating an integral R
which has no closed form representation to   The highly nonlinear form of the
constraint suggests that the standard Lagrange multiplier method will encounter
numerical diculties Indeed this was the case Moreover there is no obvious way
to study convexity and other properties of the constraint set corresponding to the
th
o contour of R
Our rst attempt is to solve the integral constraint iteratively for one of the
parameters say  in terms of the others and then choose the others to maximize
the unconstrained likelihood but with  xed That is rst choose any positive
number as a start value for   and  denoted as
c


to nd the
optimal solution for the following function for a xed
R
R
where R is given by  In other words  can be expressed as   f

a function of other three parameters Then we maximize the following loglikelihood
function l
x y for a given

l
x y
n log
n log
Xn
i
log xi
Xn
i
xi

Xn
i
log
h
exi
i
mlog   mlog
Xm
j
log yj
Xm
j
yj
Xm
j
log
h
eyj
i

We repeat the above procedure until the loglikelihood function l    given
by  is maximized We found out that this iterative method gives relative satisfactory
results when R is great than  and smaller than  on the strength
data for single carbon bre at mm mm reported by Badar and Priest
despite the computing time due to the integration The strength data are recorded
in Table  This is not surprising considering the sample sizes of  and  are
relatively large Out of this range even this method claim there is a optimal solution
at some points of R the convergence condition is not satised In statistical
software R it gives the error code  and error message of ERROR ABNORMAL
TERMINATION IN LNSRCH which indicates NaNs is produced in optimization
The mathematical software Matlab also delivers a warning message of
Matrix is singular close to singular or badly scaled Results may be inaccurate
We will show some of these results in Table  comparing with the proposed penalized
likelihood method Penalized which will be introduced in the next Section

Table  The strength data for single carbon bre at mm mm
Data Set Obs Sample Size
bre mm








bre mm









with optimal settings Table  shows that the estimated parameters using penalized
likelihood method reach higher likelihood than the iterative method Moreover
when R approaches the boundary and the sample size is getting smaller this method
runs into the singularity problem For example when n   or m   and R is
approach  or greater than  in the simulations
Table  Parameter Estimates for Example using iterative and penalized method
Method     Loglikelihood
Iterative
Penalized
Iterative
Penalized
Iterative
Penalized
Iterative
Penalized

A Penalized Likelihood Method
The penalized likelihood function PL is dened as
PL   x y  l x y  K R   N
where N is an even natural number and K  is a positive constant If K
is suciently small then it is approximately true that PL   x y  l x y
Therefore the unconstrained likelihood model is embedded in  Even when
K  is positive we still consider the unconstrained maximization of PLx y
Given that R   N is always positive as K  is set to successively larger
values the term K R   N will dominate unless the model parameters
are chosen so that K R N is very close to  and this only occurs if the
integral constraint is satised
In practice for the problems we considered we found that N could be set to
and that K  could be set to  for       and that a value of
K     worked well for the remaining more extreme values of   It is
sometimes noted in the optimization literature that a sequence of K  values may
help convergence Table  presents the parameter estimates with K
and   varying from  to  when N is chosen to be   and  Note that
when N   it corresponds to the normal MLE Table  presents the parameter
estimates with N   and   varying from  to  when K  is chosen to be

and
The nal consideration is that the Hessian matrix of PL   x y with respect
to the parameters  must be negative denite at the optimum In all cases we found
that this condition was satised In particular all of the eigenvalues of the Hessian
matrix were negative at all optima
In practice we obtained parameter estimates through unconstrained optimization
of PL using the optim function in the statistical software package R Within optim
we adopted the LBFGSB algorithm LBFGSB was developed by Byrd et al
and is a quasiNewton optimization variant of the BroydonFletcherGoldfarb
Shanno BFGS algorithm LBFGSB conveniently allows for simple upper and
lower search bounds on the parameters

Table  Parameter Estimates for Example when N    and
N     Loglikelihood
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N

Table  Parameter Estimates for Example when K     and
with N
K      Loglikelihood
K
K
K
K
K
K
K
K
K
K
K
K
K
K
K
K
K
K
K
K
K
K
K
K
K
K
K

Application to StressStrength Reliability
Inferences for the stressstrength reliability R  PY  X where X and Y are
independently distributed are a subject of interest in statistical literature and have
been widely studied in many areas Wolfe and Hogg  considered the quantity
R as a measure of dierence between distributions Hanley  discussed the importance
of R in medical applications In addition R has been examined extensively
in many other areas such as radiology reliability and material science This chapter
will discuss the inference for R  PY  X in two cases that is when X and Y are
independent Burr type X distribution and when X and Y are independent Exponentiated
Exponential distribution EE The reason of choosing these two distributions
will be reviewed in Section  and  A penalized likelihood method proposed
in Chapter  is adopted to deal with the numerical complications of maximizing the
constrained likelihood model with diering scale parameters for the two distributions
mentioned above Then following the general procedure of thirdorder method discussed
in previous Section  the detailed calculation of obtaining the inference for

R  PY  X with Burr type X is presented Section  and with Exponentiated
Exponential distribution is presented in Section  To illustrate the accuracy of
thirdorder method a real life example and simulation study are presented
Inference for StressStrength Reliability with Burr Type
X Distributions
In this section we will brie
y introduce the Burr type X distribution and present how
thirdorder method applied to Burr type X distribution We will rst consider the
case that the two independent Burr type X distribution have equal scale parameter
Then we discuss the dierent scale parameters case which is not commonly discussed
in literature Finally numerical results for both cases are presented as well
Burr type X Distribution
Burr introduced Burr type X distribution in  and following the notation in
Section  let Y be distributed as a Burr type X distribution BurrX  where
is the shape parameter and  is the scale parameter For convenience the distribution
function is rewritten here as
Fy

ey

y

BurrX  distribution is a generalized Rayleigh distribution with no shift parameter
It is also equivalent to the exponentiated Weibull      distribution as
introduced in Mudholkar and Srivastava  where  is the rst shape parameter
is the second shape parameter and  is the scale parameter Moreover if
it reduced to the Weibull   model Furthermore BurrX  is a useful model
for lifetime data because when    the model has a monotone increasing hazard
function and when    then it has bathtub hazard function Its cumulative
distribution function and survival function can be explicitly written in closed form
Various aspects of BurrX  model were studied by Sartawi and AbuSalih
Jaheen   and Ahmad et al  Several interesting properties
of the BurrX  model were discussed by Surles and Padgett  and
Raqab and Kundu  Raqab and Kundu  also studied the relations of
BurrX  model with gamma Weibull generalized exponential and exponentiated
Weibull distributions Raqab and Kundu  compared several methods of
estimating for R when X and Y followed independent Burr type X distributions
with same scale parameters Surles and Padgett   derived inference procedures
for R when X and Y are independently distributed as BurrX  and
BurrX
The Burr type X distribution is generally dicult to work with since it is neither

a transformation family model nor an exponential family model Moreover it does
not have closedform for moments or moment generating function although some
approximated moments can be obtained using results of Mudholkar et al  The
case of equal scale parameters has been extensively study in literature see Surles and
Padgett   And it is often stated in literature that the methodology can be
extended to the non equal scale parameters case However when the scale parameters
are not equal computation problems arised for any likelihoodbased method because
of the optimization problem for the constrained MLE with the constraint being an
integral without closedform The penalized likelihood method discussed in chapter
is applied to obtain the constrained MLE The proposed likelihoodbased thirdorder
method is then applied to obtain inference for R  PY  X
StressStrength Reliability with Equal Scale Parameters
Let X and Y be independently distributed as BurrX  and BurrX
respectively When the scale parameters are unequal    the stressstrength
reliability R is a form of integral in  An exact inference method for R is
not available but Surles and Padgett  obtained asymptotic inference results
for R based on the expected Fisher information matrix which will be discussed in
next subsection  When scale parameters are equal      R  R

can be simplied to R


Let x  x     xn and y  y     ym be
the random samples from BurrX  and BurrX  respectively The loglikelihood
function of the above model can be written as
l   x y  n log   mlog   n  m log
Xn
i
log xi
Xm
j
log yj


Xn
i
x
i
Xm
j
y
j


Xn
i
log  exi

Xm
j
log  eyj

Denote the overall maximum likelihood estimate as      then observed
information matrix j can be obtained as
j

BBBBBBBBB
n



Xn
i
x
i exi
exi

m



Xm
j
y
j eyj
eyi

Xn
i
x
i exi
exi
Xm
j
y
j eyj
eyi j

CCCCCCCCCA

where j
n  m


Xn
i
x
i exi   exi   x
i
exi


Xn
i
x
i
Xm
j
y
j


Xm
j
y
j eyj    eyj    y
j

eyj
The parameter of interest is


Now we dene the tilted loglikelihood
function l  as
l
n log   mlog   n  m log
Xn
i
log xi
Xm
j
log yj

Xn
i
x
i
Xm
j
y
j


Xn
i
log  exi

Xm
j
log  eyj



Then penalized likelihood method is adopted to obtain the constrained MLE
Once   is obtained we can calculate  the Lagrange multiplier by
solving



R


Hence the tilted loglikelihood function can be obtained by  Similarly the
constrained observed information matrix j  is obtained as
j


BBBBBBBBB
n









Xn
i
x
i exi
exi



m






Xm
j
y
j eyj
eyi

Xn
i
x
i exi
exi
Xm
j
y
j eyj
eyi
j


CCCCCCCCCA

where j
n  m


Xn
i
x
i exi   exi   x
i
exi


Xn
i
x
i
Xm
j
y
j


Xm
j
y
j eyj    eyj    y
j

eyj
Now we have everything to calculate r  The next step is to nd the ancillary
direction V  First let w  x     xn y     ym be the observed data and z
log  ex      log  exn  log  ey      log
eym be the vectorpivotal quantity Note that z is continuously dierentiable
and has oneone mappings between zk yk for each k Then
zi
wi


xi exi
exi  if   i  n
yin eyin
eyin  if n    i  n  m


and
zi
wj
for all i  j Hence we have the ancillary direction V
V  V V V


BBBBBBBBBBBBBBBBBBB

ex log  ex
x ex
x





exn log  exn
xn exn
xn


ey log  ey
y ey
y





eym log  eym
ym eym
ym


CCCCCCCCCCCCCCCCCCCA

Thus the locally dened canonical parameter from  is


nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi


where

l
wi


xi
xi
xi exi
exi  if   i  n
l
wi


yin
yin
yin eyin
eyin  if n    i  n  m
and its derivative is





BBBBBBBBB
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi

CCCCCCCCCA


Finally


R

R

R



Therefore  can be calculated from  q  can be calculated from  r
can be obtained from  and the
condence interval for R can be
obtained Since the results from BN  and LN  are almost identical we
will only consider BN hereafter
Here we analyze the strength data for single carbon bre reported by Badar
and Priest  Badar and Priest reported the strength data for single carbon
bre at mm mm mm and mm gauge length which is given in Table
Surles and Padgett   drew inferences for R  PY  X on mm
and mm gauge lengths where X represent the strength of mm ber and Y
represents the strength of mm ber with sample sizes of  and  respectively
Assuming X and Y follow Burr type X distributions with equal scale parameters
Surles and Padgett  reported the MLE of R is R    and concluded
that R is greater than  using the approximate inference procedures they suggested
This is consistent with the fact of that short bers tends to be stronger than long
ones
Assume that X and Y are independently distributed as BurrX  and BurrX
respectively We are interested in testing H  R   vs H  R   where

R  PY  X is given in  Using the techniques discussed in Chapter  the
pvalues are   and  obtained by the Wald method the signed
loglikelihood ratio method and the proposed method respectively At the  signi
cance level the proposed thirdorder method gives dierent conclusion than the
other two rstorder methods That is proposed thirdorder method fail to reject
H at  level whereas the rstorder methods reject H at  level Moreover
the  and  condence intervals CI for R based on Wald r and proposed
thirdorder Proposed intervals are presented in Table  In examining Table
we observe that all three methods give similar interval estimations The result is not
surprising because both samples are relatively large
Table  Interval Estimates of   for Example assuming equal scale parameter
Condence Interval  Condence Interval
Wald
r
Proposed
To examine the accuracy of the methods discussed in this dissertation simulation
studies are performed To compare the accuracy of the proposed method with the
Wald method and the signed loglikelihood ratio method Monte Carlo simulation
studies were conducted
For each parameter conguration and for each sample size we generate

random samples from Burr type X distributions by using the following transformation
T


h
log  U


i

where U is a uniform variate between  and  For each simulated sample we calculated
the  condence interval for   obtained by the proposed method Proposed
with the Wald method Wald and the signed loglikelihood ratio method r For
each simulated setting we report the same comparison criterions used in Chapter
lower error upper error central coverage and the average bias The nominal values
for the lower and the upper errors the central coverage and the average bias are
and  respectively These values re
ect the desired properties of
the accuracy and symmetry of the interval estimates of R
Tables   and  present the simulation results for the cases that
More specically we present results using the parameters setting
and R   with nm      and   Note that
is uniquely determined from R

Table          nm    and  satises R
R Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed

Table          nm    and  satises R
R Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed

Table          nm    and  satises R
R Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed

It is clear that the coverage probabilities for R are poor and the twotail error
probabilities are extremely asymmetric from the Wald method in all three settings
The results from signed loglikelihood method are not satisfactory The proposed
method gives not only an almost exact coverage probability but also it has symmetric
twotail error probabilities even for small sample sizes
StressStrength Reliability with Unequal Scale Parameters
When scale parameters are unequal    R in  becomes dicult to calculate
since integral has no known closedform Let x  x     xn and y  y     ym
be the random samples from BurrX  and BurrX  respectively By
maximizing the loglikelihood function in  the overall MLE can be obtained
Denote the overall MLE as       The observed information matrix
evaluated at  is
j

BBBBBBBBBBBBBB
n



Xn
i
x
i exi
exi

m



Xm
j
y
j e  yj
eyi

Xn
i
x
i exi
exi  j

Xm
j
y
j e  yj
eyi  j

CCCCCCCCCCCCCCA


where j
n


Xn
i
x
i
Xn
i
x
i exi   exi
x
i
exi
and j
m


Xm
j
y
j
Xm
j
y
j e  yj    eyj
y
j

eyj
The constrained MLE        for a given   is obtained by penalized
likelihood method discussed in Chapter  subject to the constraint R
Once   is obtained we can calculate  the Lagrange multiplier by solving



R


Hence the tilted loglikelihood function can be obtained by  Thus the observed
information matrix for the tilted loglikelihood function evaluated at constrained
MLE   is j   l  and can be written as
j


BBBBBBBBBB
j
j  j  j
j
j  j  j
j
j  j  j
j
j  j  j

CCCCCCCCCCA
where
j
n


R  where R
R



j    R  where R
R



j
Xn
i
x
i exi
exi   R  where R
R




j    R  where R
R



j
m


R  where R
R



j    R  where R
R



j
Xm
j
y
j eyj
eyi   R  where R
R



j
n


Xn
i
x
i
Xn
i
x
i exi   exi
x
i
exi
R  where R
R



j    R  where R
R



j
m


Xm
j
y
j
Xm
j
y
j e  yj    eyj
y
j

eyj
R  where R
R



Now let w  x     xn y     ym be the observed data and z  z     zn zn     znm
denoted the vectorpivotal quantity in this case z  log Fx       log Fxn
log Fy       log Fym   and
zi
wi




xi exi
exi  if   i  n

yin eyin
eyin  if n    i  n  m


and
zi
wj
for all i  j Hence from  we have
V  V V V V


BBBBBBBBBBBBBBBBBBB

log  ex  ex

x ex
x






log  exn  exn

xn exn
xn



log  ey  ey

y ey
y






log  eym  eym

ym eym
ym


CCCCCCCCCCCCCCCCCCCA
Thus the locally dened canonical parameter from  is


nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi


where


l
wi


xi

xi

xi exi
exi  if   i  n
l
wi


yin

yin

yin eyin
eyin  if n    i  n  m

and its derivative is





BBBBBBBBBBBBBB
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi

nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi

nXm
i
l
wi
Vi
nXm
i
l
wi
Vi

CCCCCCCCCCCCCCA

Finally


R

R

R

R


Therefore q  can be calculated from  r  can be obtained from  and
the
condence interval for R can be obtained
To illustrate the proposed method with unequal scale parameters we will use
the same data sets of carbon bre reported by Badar and Priest  Surles and
Padgett  reported the MLE of R is R    and concluded that R is
greater than  using the approximate inference procedures they suggested This is
consistent with the fact of that short bers tends to be stronger than long ones
Again assuming that X and Y are independently distributed as BurrX
and BurrX  respectively We are interested in testing H  R   vs

H  R   where R  PY  X is given in  Using the techniques discussed
in Section  the pvalues are   and  obtained by the
Wald method the signed loglikelihood ratio method and the proposed method respectively
They provide strong evidence to suggest that mm ber is typically
stronger than mm ber
Moreover the  and  condence intervals CI for R based on Wald r
and proposed thirdorder method Proposed intervals are presented in Table  In
examining Table  we observe that all three methods give similar interval estimations
The result is not surprising because both samples have relatively large sample
sizes
Table  Interval Estimates of   for Example assuming unequal scale parameters
Condence Interval  Condence Interval
Wald
r
Proposed
We randomly sampled from this data set using n    and n    for
each combination of nm we obtained the  and  CI for R and the results
are reported in Table  As we can see in Table  the proposed method and the
other methods give quite dierent results for small sample size situations
We also conduct the Monte Carlo simulation studies when the scale parameters

Table  Interval Estimates of   for Example using resampling
Sample sizes Methods  Condence Interval  Condence Interval
Wald
n m r
Proposed
Wald
n m r
Proposed
of two Burr type X distributions are dier with similar settings comparing when
scale parameter is same For each parameter conguration and for each sample
size we generate  random samples from Burr type X distribution by using
the transformation from  For each simulated sample we calculated the
condence interval for   obtained by the proposed thirdorder method with the Wald
method and the signed loglikelihood ratio method For each simulated setting we
report the same criterions lower error upper error central coverage and the average
bias
Table  presents the simulation results of  lower errors upper errors coverage
probabilities and average bias by choosing       and
which is closed to the overall MLE obtained from example in the previous section
when the same sample sizes are equal n  m   and when sample sizes are not
equal n   m   and n   m   In this case R      It is clear

Table  Simulation results for R when  is not equal
Sample Size Method Lower Error Upper Error Central Coverage Average Bias
Wald
nm r
Proposed
Wald
n m r
Proposed
Wald
n m r
Proposed
that the coverage probabilities for R are poor and the twotail error probabilities are
extremely asymmetric from the Wald method in all three settings The results from
signed loglikelihood method are not satisfactory The proposed method gives not
only an almost exact coverage probability but also it has symmetric twotail error
probabilities even for small or uneven sample sizes
Tables  to  present more simulation results for          and
R   with nm      and   Note that we xed R
and  is determined uniquely by  Again the proposed method outperformed
the other two methods even when the sample sizes are small or uneven

Table              and  is obtained by Equation
nm
R Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed

Table              and  is obtained by Equation
nm
R Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed

Table              and  is obtained by Equation
nm
R Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed

Inference for StressStrength Reliability with Exponen
tiated Exponential Distribution
Gupta et al  and Gupta and Kundu  introduced a twoparameter exponentiated
exponential EE distribution as an alternative to the twoparameter
gamma and Weibull distributions for analysing failure time data For the twoparameter
gamma distribution its cumulative distribution function or equivalently
the survival function cannot be expressed in closed form if the shape parameter is
not an integer This makes it less popular than the Weibull distribution whose
distribution function survival function and hazard function can all be expressed
as a closed form However for the Weibull distribution the asymptotic coverage
to normality for the distribution of the maximum likelihood estimators MLEs is
very slow Bain  Therefore most of the asymptotic inferences are not very
accurate unless the sample size is very large Compare to these two most commonly
used distributions the cumulative distribution function of EE distribution can be
expressed in explicit form share some common properties with gamma and Weibull
distributions for failure time data and has fast convergence of the MLEs
The distribution function Fy   of the EE distribution takes the form
Fy      ey   y

where  is the shape parameter and  is the scale parameter
Similar to the twoparameter gamma distribution the EE distribution also has an
increasing or decreasing failure rate that depends on the shape parameter The density
function of the EE distribution also varies signicantly depending on the shape
parameter The advantage of the EE distribution is that the cumulative distribution
function and the survival function can be explicitly written in closed form
StressStrength Reliability with Equal Scale Parameters
Let X and Y be independently distributed as EE  and EE  respectively
Then the stressstrength reliability with same scale parameter  is
R  PY  X
Z

Z x

fx   fy   dy dx




Let x  x     xn and y  y     ym be the random samples from EE
and EE  respectively Then the loglikelihood function of above model can be
written as
l   x y  n log   mlog   n  m log

Xn
i
xi
Xm
j
yj


Xn
i
log  exi
Xm
j
log  eyj

Denote the overall maximum likelihood estimate as       then the observed
information matrix j can be obtained as
j  l


BBBBBBBBB
n



Xn
i
xi
exi

m



Xm
j
yj
eyj

Xn
i
xi
exi

Xm
j
yj
eyj
n  m

A

CCCCCCCCCA

where A
Xn
i
x
i exi
exi

Xm
j
y
j eyj
eyj

The tilted loglikelihood function l is dened as
l
lx y
where    R


Then the constrained MLE can be obtained from
penalized likelihood method and denote as         can be calculated by
Lagrange multiplier method and constrained observed information matrix j
can be written as
j


BBBBBBBBB
n









Xn
i
xi
exi



m






Xm
j
yj
eyj

Xn
i
xi
exi

Xm
j
yj
eyj
n  m

B

CCCCCCCCCA


where B
Xn
i
x
i exi
exi

Xm
j
y
j eyj
eyj
Now we have
everything to calculate r
Let z  z     zn zn     znm denotes the vectorpivotal quantity in this
case z  log Fx       log Fxn   log Fy
log Fym   Thus the ancillary direction V is
V  V V V

BBBBBBBBBBBBBBBBBBBB
log  ex
ex


x




log  exn
exn


xn

log  ey
ey


y



log  eym
eym


ym


CCCCCCCCCCCCCCCCCCCCA
Then we can calculate the locally dened canonical parameter


nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi

where w  x     xn y     ym be the observed data Hence we also have
For this particular case  can be obtained accordingly Therefore var




Q  and r  can be obtained Hence
condence interval can be obtained
from the modied signed loglikelihood ratio statistics
To illustrate the proposed thirdorder method for interval estimation the follow

ing two data sets with sample size of  and  are used
and
Assume that X and Y are independently
distributed as EE  and EE  respectively We are interested
in testing H  R   vs H  R   where R  PY  X is given in
The  and  condence intervals for R based on Wald r and r methods are
presented in Table  In examining Table  we observe that all three methods
give dierent interval estimations
Table  Interval Estimates of   Assuming EED with Same Scale Parameter
Condence Interval  Condence Interval
Wald
r
Proposed
To compare the accuracy of the proposed method with the MLE method the
signed loglikelihood ratio method and the proposed method Monte Carlo simulation
studies were conducted The cases of unequal and equal scale parameters
are both examined For each parameter conguration and for each sample size we
generate  random samples from EE distribution by using the following transformation
T


log  U

where U is a uniform variate between  and
The performance of a method is judged by using the same criteria as before
central coverage upper error lower error and average bias These values re
ect the
desired properties of the accuracy and symmetry of the interval estimates of R
Tables  to  present simulation results for the equal scale parameter case
ie       and R   with nm      and
Note that we xed R and  is determined uniquely by R     Again
the proposed method outperformed the other two methods even when the sample
sizes are small

Table        and  satises R     nm
R Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed

Table        and  satises R     nm
R Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed

Table        and  satises R     nm
R Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed

StressStrength Reliability with Unequal Scale Parameters
Let X and Y be independently distributed as EE  and EE  respectively
When scale parameters are unequal    R is dicult to calculate since
integral has no known closedform which is
R  PY  X
Z

ex ex   ex dx
Exact inference procedure for R is not available Let x  x     xn and y
y     ym be the random samples from EE  and EE  respectively
Then the loglikelihood function of above model can be written as
l    x y  n log   n log
Xn
i
log  exi

Xn
i
xi  mlog   mlog
Xm
j
log  eyj
Xm
j
yj
Denote the overall maximum likelihood estimate as         and observed
information matrix j can be obtained as
j


BBBBBBBBBBBBBB
n



Xn
i
xi exi
exi


Xn
i
xi exi
exi
n


A

m



Xm
j
yj eyj
eyj

Xm
j
yj eyj
eyj
m


B

CCCCCCCCCCCCCCA

where A
Xn
i
x
i exi
exi
and B
Xm
j
y
j eyj
eyj

The tilted loglikelihood function l is dened as
l
lx y
where    R dened by  Similarly we can obtain the constrained MLE
by penalized likelihood method and  by Lagrange multiplier
method then constrained observed information matrix j  can be written as
j
l

BBBBBBBBBB
j
j  j  j
j
j  j  j
j
j  j  j
j
j  j  j

CCCCCCCCCCA
where
j
n


R  where R
R





j    R  where R
R




j
Xn
i
xi exi
exi
R  where R
R




j    R  where R
R




j
m

R  where R
R






j    R  where R
R




j
Xm
j
yj eyj
eyj
R  where R
R




j
m



Xn
i
x
i exi
exi
R  where R
R





j    R  where R
R




j
m


Xm
j
y
j e  yj
eyj
R  where R
R





Thus r  can be obtained accordingly
Let z  z     zn zn     znm denotes the vectorpivotal quantity in this
case z  log Fx       log Fxn   log Fy

log Fym   Hence the ancillary direction V is
V  V V V V


BBBBBBBBBBBBBBBBBBBB
log  ex
ex
ex

x





log  exn
exn
exn

xn


log  ey
ey
ey

y





log  eym
eym
eym

ym


CCCCCCCCCCCCCCCCCCCCA
Then we can calculate the locally dened canonical parameter  as


nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi
nXm
i
l
wi
Vi

where w  x     xn y     ym be the observed data Hence we also have
Therefor for this unequal scale parameter case  can be obtained accordingly
Therefore var



Q  and r  can be obtained Hence

condence interval can be obtained from the modied signed loglikelihood ratio
statistics
Table  present the  and  condence intervals CI for R based on
Wald r and proposed thirdorder methods by using the same data set recorded in
Section

Table  Interval Estimates of   for Example
Condence Interval  Condence Interval
Wald
r
Proposed
We also conduct Monte Carlo simulation studies The performance of a method
is judged by using the following same criteria central coverage upper error lower
error and average bias
Tables  to  present simulation results for the unequal scale parameters
case ie          and R   with nm
and   Note that we xed R and  is determined uniquely by Equation
It is clear that the coverage probabilities for R are poor and the twotail
error probabilities are extremely asymmetric from the Wald method The results
from signed loglikelihood method are not satisfactory especially when two sample
sizes are small or sample sizes are unequal and it also shows some evidence of
asymmetry of twotail error probabilities However the proposed method gives not
only an almost exact coverage probability but also it has symmetric twotail error
probabilities even for small or uneven sample sizes

Table              and  is obtained by Equation
nm
R Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed

Table              and  is obtained by Equation
nm
R Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed

Table              and  is obtained by Equation
nm
R Method Lower Error Upper Error Central Coverage Average Bias
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed
Wald
r
Proposed

Conclusion and Future Work on Thirdorder Asymp
totic Methods
A likelihoodbased higher order asymptotic method is proposed to apply inference
in the stressstrength reliability model when the two populations are distributed
independently When the scale parameters are dierent the constrained MLE cannot
be obtained by standard Lagrange multiplier methods and a penalized likelihood
method is proposed The proposed thirdorder method exhibits high accuracy and
the penalized likelihood method can be easily employed for relatively small data set
Based on the current work there are several possible directions that research
could be extended to
Theoretically the proposed method can be applied to obtain inference for
stressstrength reliability from any distributions In this dissertation the parameter
is from a continuous distribution However when the distribution is
of a discrete nature the ancillary direction V cannot be obtained by dierentiation
It will be important if an appropriate approach can be developed
The thirdorder method results in extremely accurate pvalues and condence
intervals In applied work researchers may deal with inference for hierarchical
structure data for example in medical science Extending thirdorder methods

to those elds will be an interesting future project
The exponential approximation plays an important role of providing accurate
approximations to general statistical models in thirdorder inference The is
due to that p does not dependent on the nonexponential term to the third
order and it depends only on the observed likelihood and the gradient of the
likelihood at the data point This dissertation illustrates that this property
hold for univariate model When working with multivariate or non independent
situation data does this property still hold This topic remains challenging

Bibliography
Ader H J Mellenbergh G J and Huizen D J H  Advising on research
methods A consultants companion Journal of Applied Statistics
Agrawal R Gollapudi S Halverson A and Ieong S a Diversifying search
results In Proceedings of the Second ACM International Conference on Web Search
and Data Mining WSDM  pages
Agrawal R Gollapudi S Halverson A and Ieong S b Diversifying search
results In Proceedings of the Second ACM International Conference on Web Search
and Data Mining pages
Ahmad K Fakhry M and Jaheen Z  Empirical bayes estimation of py
x and characterizations of the burrtype x model Journal of Statistical Planning
and Inference
Badar M and Priest A  Statistical aspects of bre and bundle strength

in hybrid composites Progress in Science and Engineering CompositesT Hayashi
KKawata and S Umekawa eds ICCMIV pages
Bain L  Statistical analysis of reliability and life testing model Marcel and
Dekker Inc New York
BarndorNielsen O E  Information and Exponential Families in Statistical
Theory Wiley Series in probability and mathematical statistics John Wiley  Sons
BarndorNielsen O E  Conditionality resolutions Biometrika
BarndorNielsen O E  On a formula for the distribution of the maximum
likelihood estimate Biometrika
BarndorNielsen O E  Inference on full or partial parameters based on
the standardized signed log likelihood ratio Biometrika
BarndorNielsen O E  Modied signed loglikelihood ratio statistic
Biometrika
BarndorNielsen O E  Inference and Aspptotics London Chapman and
Hall
BarndorNielsen O E and Chamberlin S R  An ancillary invariant modi

cation of the signed log likelihood ratio Scandinavian Journal of Statistics

BarndorNielsen O E and Cox D R  Edgeworth and saddlepoint approximations
with statistical applications with discussion Journal of the Royal
Statistical Society B
BarndorNielsen O E and Cox D R  Bartlett adjustments to the likelihood
ratio statistic and the distribution of the maximum likelihood estimator Jour
nal of the Royal Statistical Society B
BarndorNielsen O E and Cox D R  Asymptotic techniques for use in
statistics Metrika
Box G E P and Cox D R  An analysis of transformations Journal of the
Royal Statistical Society Series B Methodological
Bttcher S and Clarke C L A  Eciency vs eectiveness in terabytescale
information retrieval In TREC volume Special Publication
Buckley C and Robertson S E editors  Relevance feedback track overview
TREC

Burr I W  Cumulative frequency functions Annuals of Mathematical Statis
tics
Byrd R Lu P Nocedal J and Zhu C  A limited memory algorithm for
bound constrained optimization SIAM J Scientic Computing
Cao Y Liu J Bao S and Li H  Research on expert search at enterprise
track of trec  volume Special Publication
Carbonell J and Goldstein J  The use of mmr diversitybased reranking for
reordering documents and producing summaries In In Research and Development
in Information Retrieval pages
Carpineto C de Mori R Romano G and Bigi B  An informationtheoretic
approach to automatic query expansion ACM Trans Inf Syst
Clarke C L Craswell N and Soboro I a Overview of the trec  web
track Technical report
Clarke C L Craswell N and Soboro I b Preliminary report on the trec
web track In Proc of TREC
Clarke C L Kolla M Cormack G V Vechtomova O Ashkan A Buttcher S
and MacKinnon I a Novelty and diversity in information retrieval evaluation

In Proceedings of the st annual international ACM SIGIR conference on Research
and development in information retrieval SIGIR  pages
Clarke C L Kolla M Cormack G V Vechtomova O Ashkan A Buttcher S
and MacKinnon I b Novelty and diversity in information retrieval evaluation
In Proceedings of the st annual international ACM SIGIR conference on Research
and development in information retrieval pages
Cooper W  The Suboptimality of Retrieval Rankings Based on Probability of
Usefulness School of Library and Information Studies
Cox D R and Oakes D  Analysis of Survival Data
Craswell N Fetterly D Najork M Robertson S and Yilmaz E  Microsoft
research at trec  Web and relevance feedback track In TREC volume
Special Publication
Croft W B Turtle H R and Lewis D D  The use of phrases and
structured queries in information retrieval In Proceedings of the th Annual In
ternational ACM SIGIR Conference on Research and Development in Information
Retrieval Chicago Illinois USA October   Special Issue of the SIGIR
Forum pages  ACM

Daniels H E  Saddlepoint Approximations in Statistics The Annals of
Mathematical Statistics
DemnerFushman D Humphrey S M Ide N C Loane R F Mork J G Ruch
P Ruiz M E Smith L H Wilbur W J and Aronson A R  Combining
resources to nd answers to biomedical questions In TREC
DemnerFushman D and Lin J  Answering clinical questions with
knowledgebased and statistical techniques Comput Linguist
Doganaksoy N and Schmee J  Comparisons of approximate condence
intervals for distributions used in lifedata analysis Technometrics
Durbin J  Approximations for densities of sucient estimators Biometrika

Engle R F  Handbook of Econometrics volume Volume  chapter Wald
likelihood ratio and Lagrange multiplier tests in econometrics pages  Elsevier
Fagan J L  Automatic phrase indexing for document retrieval An examination
of syntactic and nonsyntactic methods In SIGIR pages  ACM

Fisher R A  On the probable error of a coecient of correlation deduced
from a small sample Metron
Fisher R A  On the mathematical foundations of theoretical statistics
Philosophical Transactions of the Royal Society of London A
Fraser D A S  Normed likelihood as saddlepoint approximation Multivari
ate Analysis
Fraser D A S  Tail probabilities from observed likelihoods Biometrika

Fraser D A S and Reid N  Third order asymptotic models likelihood
functions leading to accurate approximations for distribution functions Statistica
Sinica
Fraser D A S and Reid N  Ancillaries and third order signicance Utilitas
Mathematics pages
Fraser D A S and Reid N  Ancillary information for statistical inference
Technical report
Fraser D A S Reid N and Wong A  Exponential linear model a two

pass procedure for saddlepoint approximation Journal of Royal Statistical Society
B
Fraser D A S Reid N and Wong A  pvalue formulas from likelihood
asymptotics bridging the singularities Journal of Statistical Research
Fraser D A S Reid N and Wu J  A simple general formula for tail
probabilities for frequentist and bayesian inference Biometrika
Gao J Nie JY Wu G and Cao G  Dependence language model for
information retrieval In Proceedings of the th annual international ACM SIGIR
conference on Research and development in information retrieval SIGIR  pages

Gentle J E  Elements of computational statistics Springer
Gobeill J Tbahriti I Ehrler F and Ruch P  Vocabularydriven passage
retrieval for questionanswering in genomics In Proceedings of the th Text
REtrieval Conference TREC Maryland USA
Gumbel E J  Statistics of extremes Columbia University Press New York
Gupta R Gupta P and Gupta R  Modeling failure time data by lehman
alternatives Communications in Statistics  Theory and Methods

Gupta R and Kundu D  Exponentiated exponential family an alternative
to gamma and weibull distributions Biometrical Journal
HancockBeaulieu M Gatford M Huang X Robertson S E Walker S and
Williams P W  Okapi at trec
Hanley J  Receiver operating characteristicroc methodology the state of
the art Critical Reviews in Diagnostic Imaging
He B Huang J X and Zhou X  Modeling term proximity for probabilistic
information retrieval models Information Sciences
Hersh W Cohen A Ruslen L and Roberts P  TREC  Genomics
Track Overview In In Proceedings of the Text REtrieval Conference TREC
Hersh W Cohen A M Roberts P and Rekapalli H K a TREC
genomics track overview
Hersh W Cohen A M Roberts P and Rekapalli H K b Trec
genomics track overview In In Proceedings of the Text REtrieval Conference TREC
Hogg R V Craig A and Mckean J W  Introduction to Mathematical
Statistics Prentice Hall th edition

Ide N C Loane R F and DemnerFushman D  Essie a conceptbased
search engine for structured biomedical text Journal of American Medical Infor
matics Association
Jaheen Z  Bayesian approach to prediction with outliers from the burr type
x model Microelectronics Reliability
Jaheen Z  Empirical bayes estimation for the reliability and failure rate
function of burr type x failure model Journal of Applied Statistical Science

Jensen J L  The modied signed likelihood statistic and saddlepoint approximations
Biometrika
Kaptein R Koolen M and Kamps J  Result diversity and entity ranking
experiments Anchors links text and wikipedia In TREC volume Special Publication

Kirkpatrick S Gelatt C D and Vecchi M P  Optimization by simulated
annealing Science
Kraaij W Westerveld T and Hiemstra D  The importance of prior
probabilities for entry page search SIGIR  pages

Lavrenko V and Croft W B  Relevance based language models In Pro
ceedings of the th annual international ACM SIGIR conference on Research and
development in information retrieval pages
Li J and Yan H  Peking university at the trec  terabyte track volume
Special Publication
Losada D E and Azzopardi L  An analysis on document length retrieval
trends in language modeling smoothing Inf Retr
Losee R M and Jr  Term dependence Truncating the bahadur lazarsfeld
expansion In Information Processing and Management pages
Lugannani R and Rice S  Saddlepoint approximation for the distribution
function of the sum of independent variables Advanced Applied Probability

Lv Y and Zhai C  Positional language models for information retrieval
In Proceedings of the nd international ACM SIGIR conference on Research and
development in information retrieval SIGIR  pages
McCreadie R Macdonald C Ounis I Peng J and Santos R L T
University of glasgow at trec  Experiments with terrier In TREC

McCullagh P  Tensor methods in statistics Monographs on statistics and
applied probability Chapman and Hall London ua
Metzler D and Croft W B a A markov random eld model for term dependencies
In Proceedings of the th annual international ACM SIGIR conference
on Research and development in information retrieval SIGIR  pages
Metzler D and Croft W B b A markov random eld model for term dependencies
In Proceedings of the th annual international ACM SIGIR conference
on Research and development in information retrieval pages
Metzler D Strohman T and Croft W B  Indri trec notebook
Lessons learned from three terabyte tracks volume Special Publication
Mishne G and de Rijke M  Boosting web retrieval through query operations
In ECIR volume  pages
Mudholkar G and Srivastava D  Exponentiated weibull family for analyzing
bathtub failurerate data IEEE Transactions on Reliability
Mudholkar G Srivastava D and Freimer M  The exponentiated weibull
family A reanalysis of the busmotorfailure data Technometrics
Neyman J and Pearson E S  On the problem of the most ecient tests

of statistical hypotheses Philosophical Transactions of the Royal Society of London
Series A Containing Papers of a Mathematical or Physical Character
Ounis I Amati G Plachouras V He B Macdonald C and Lioma C a
Terrier A High Performance and Scalable Information Retrieval Platform In Pro
ceedings of ACM SIGIR Workshop on Open Source Information Retrieval OSIR

Ounis I de Rijke M Macdonald C Mishne G and Soboro I b
Overview of the trec blog track In Text Retrieval Conference
Plachouras V and Ounis I a Multinomial randomness models for retrieval
with document elds In ECIR volume  pages
Plachouras V and Ounis I b Multinomial randomness models for retrieval
with document elds In ECIR volume  pages  Springer
Rao C R  Large sample tests of statistical hypotheses concerning several
parameters with applications to problems of estimation Mathematical Proceedings
of the Cambridge Philosophical Society
Raqab M and Kundu D  Comparison of dierent estimators of py  x
for a scaled burr type x distribution Communications in Statistics  Simulation and
Computation

Raqab M and Kundu D  Burr type x distribution Revisited Journal of
Probability and Statistical Sciences
Rasolofo Y and Savoy J  Term proximity scoring for keywordbased retrieval
systems In In Proc of the th European Conf on IR Research pages
Reid N  Saddlepoint Methods and Statistical Inference Statistical Science

Reid N  Likelihood and higherorder approximation to tail areas a review
and annotated bibliography Canadian Journal of Statistics
Robertson S Walker S Jones S HancockBeaulieu M and Gatford M
Okapi at trec pages
Robertson S E  The probability ranking principle in ir Journal of Docu
mentation
Robertson S E and Jones S K  Relevance weighting of search terms
Journal of the American Society for Information Science
Robertson S E van Rijsbergen C J and Porter M F  Probabilistic
models of indexing and searching In Proceedings of the rd annual ACM conference
on Research and development in information retrieval SIGIR  pages

Robertson S E and Walker S  Some simple eective approximations to the
poisson model for probabilistic weighted retrieval In Proceedings of the th annual
international ACM SIGIR conference on Research and development in information
retrieval SIGIR  pages  New York NY USA SpringerVerlag New York
Inc
Robertson S E Walker S HancockBeaulieu M Gatford M and Payne A
Okapi at trec In TREC
Santos R L Macdonald C and Ounis I a Exploiting query reformulations
for web search result diversication In Proceedings of the th international
conference on World wide web WWW  pages
Santos R L T Peng J Macdonald C and Ounis I b Explicit search
result diversication through subqueries In ECIR volume  pages
Sartawi H and AbuSalih M  Bayesian prediction bounds for the burr type
x model Communications in StatisticsTheory and Methods
Schechtman E and Sherman M  The twosample ttest with a known ratio
of variances Statistical Methodology
Shao J  Mathematical Statistics SpringerVerlag New York Inc nd edition

She Y Wong A and Zhou X  Revisit the two sample ttest with a known
ratio of variances Open Journal of Statistics
Silverman B W  Density Estimation for Statistics and Data Analysis Chapman
Hall London
Singhal A Buckley C and Mitra M a Pivoted document length normalization
In Proceedings of the th annual international ACM SIGIR conference on
Research and development in information retrieval SIGIR  pages
Singhal A Salton G Mitra M and Buckley C b Document length
normalization Inf Process Manage
Song R Wen JR Shi S Xin G Liu TY Qin T Zheng X Zhang J
Xue GR and Ma WY  Microsoft research asia at web track and terabyte
track of trec  In TREC volume Special Publication
Stokes N Li Y Cavedon L Huang E Rong J and Zobel J  Entitybased
relevance feedback for genomic list answer retrieval In Proceedings of The
Sixteenth Text REtrieval Conference TREC  Gaithersburg Maryland USA
November
Surles J and Padgett W  Inference for py  x in the burr type x model
Journal of Applied Statistical Science

Surles J and Padgett W  Inference for reliability and stressstrength for a
scaled burr type x distribution Lifetime Data Analysis
Surles J and Padgett W  Some properties of a scaled burr type x distribution
Journal of Statistical Planning and Inference
van Rijsbergen C J  A theoretical basis for the use of cooccurrence data
in information retrieval Journal of Documentation
Voorhees E Harman D of Standards N I and US T  TREC Exper
iment and evaluation in information retrieval volume  MIT press Cambridge
eMA MA
Voorhees E M and Buckland L P editors  Proceedings of The Eighteenth
Text REtrieval Conference TREC  Gaithersburg Maryland USA November

Wald A  Tests of Statistical Hypotheses Concerning Several Parameters
When the Number of Observations is Large Transactions of the American Mathe
matical Society
Wilks S S  The LargeSample Distribution of the Likelihood Ratio for
Testing Composite Hypotheses The Annals of Mathematical Statistics

Wolfe D A and Hogg R V  On constructing statistics and reporting data
The American Statistician
Ye Z Huang X He B and Lin H  York university at trec  Relevance
feedback track In TREC volume Special Publication
Yin X Huang J X and Li Z  Mining and modeling linkage information
from citation context for improving biomedical literature retrieval Information
Processing and Management
Yin X Huang J X Zhou X and Li Z  A survival modeling approach to
biomedical search result diversication using wikipedia In SIGIR pages
Yu C T Buckley C Lam K and Salton G  A Generalized Term Dependence
Model in Information Retrieval Technical report Cornell University
Zhai C X Cohen W W and Laerty J  Beyond independent relevance
methods and evaluation metrics for subtopic retrieval In Proceedings of the th
annual international ACM SIGIR conference on Research and development in infor
maion retrieval SIGIR  pages
Zhang Y Callan J and Minka T  Novelty and redundancy detection
in adaptive ltering In Proceedings of the th annual international ACM SIGIR

conference on Research and development in information retrieval SIGIR  pages

Zhao J and Yun Y  A proximity language model for information retrieval
In SIGIR pages
Zhou X Huang J X and He B  Enhancing adhoc relevance weighting
using probability density estimation In Proceedings of the th International ACM
SIGIR Conference on Research and Development in Information Retrieval SIGIR
pages
Zhu J Huang X Song D and Rger S M  Integrating multiple document
features in language models for expert nding Knowl Inf Syst

